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ABSTRACT 


The  behavior  of  digital  cross-correlation  algorithms 
as  applied  to  image  matching  problems  is  examined  in  terms 
of  the  relationship  between  measureable  image  properties  and 
algorithm  characteristics.  Statistical  image  quality  meas¬ 
ures  are  developed  which  could  be  employed  in  a  preprocessor 
to  predict  the  performance  of  automatic  stereo-compilation 
equipment.  The  measures  include  a  quantity  derived  from  the 
Cramer-Rao  lower  bound  on  the  variance  of  any  unbiased 
parameter  estimator,  various  contrast  measures  such  as  vari¬ 
ance,  contrast  modulation,  and  median  absolute  deviation, 
and  a  stationarity  detector  related  to  the  variance  gracient. 
Tnese  measures  are  based  on  image  and  correlator  models 
which  describe  the  behavior  of  correlation  processors  under 
conditions  of  low  image  contrast  or  signai-to-noise  ratio, 
geometric  distortion,  and  image  non-stationarity.  Computer 
simulations  using  synt.netic  imagery  were  performed  to  ver¬ 
ify  the  various  models,  and  indicate  the  potential  for  the 
use  c f  image  quality  measures  in  the  predicting  of  corre¬ 
lation  behavior.  Implications  of  tne  models  in  terms  of 
correlation  processor  design  ano  implementation  are  dis- 
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CHAPTER  1 

INTRODUCTION 

In  rrany  image  orocessing  applications,  it  is  neces¬ 
sary  to  spatially  register  two  or  more  images  of  an  object 
field  obtained  from  different  sensors  or  taken  from  the  sate 
sensor  at  different  times  or  locations.  The  registration  of 
two  images  consists  of  translating  them  relative  to  one  an¬ 
other  so  that  they  exactly  overlay  each  other.  The  transla¬ 
tion  (ana  possibly  rotation)  required  to  achieve  registra¬ 
tion  provides  information  concerning  the  relative  positions 
of  the  sensors  and  points  in  the  object  field.  Applications 
of  image  registration  are  numerous  and  include  such  diverse 
areas  as  guidance  of  unmanned  vehicles,  c range  detection, 
target  acquisition  and  tracking,  and  the  production  of  topo¬ 
graphic  maps.  It  is  primarily  the  latter  topic  that  is  ac- 
oressed  herein  although  many  of  the  concepts  and  problems 
to  be  discussed  are  also  found  in  other  applications. 

A  problem  of  considerable  interest  within  the  car¬ 
tographic  community  is  that  of  mechanised  " s te reo- v i ew i n g"  , 
that  is,  automatically  locating  identical  (conjugate)  points 
on  two  or  mere  aerial  onotographs.  The  result  of  this  in¬ 
terest  nas  been  tne  development  of  several  automated  stereo- 
compilation  systems  including  tne  Universal  Automatic  Map 
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Compilation  Equipment  (UNAMACE)  [Thompson  1966],  the  AS- 
11B-X  Stereo  Mapper  [3rumm,  et  al.  1976],  the  Gestalt  Photo 
Mapper  G D M I  I  [Crawley  1  974],  and  the  Heterodyne  Optical 
Correlator  (HOC)  [Ba 1  a  sub ranan i an  1  976  ].  The  principal 
advantage  of  automates  stereo-systems  are  their  speed, 
economy  and  precision,  (al  trough  not  necessarily  accuracy). 
Different  human  operators  using  a  stereo-plotter  will  not 
achieve  the  same  degree  of  repeatability  as  will  the  auto¬ 
mated  system.  The  human  operator,  however,  is  far  more  re¬ 
liable.  The  limitation  in  machine  reliability  is  a  function 
of  the  variability  of  terrain  conditions  over  which  it  must 
operate.  The  basic  function  of  the  automatic  stereo¬ 
compiler  is  to  compare  two  nearly  identical  images  (around 
corresponding  conjugate  points)  and  determine  the  position 
resulting  in  maximum  similarity.  The  machine,  however,  is 
unable  to  recognize  objects  or  shapes.  When  the  machine  is 
confronted  with  images  that  differ  considerably  in  appear¬ 
ance,  it  does  not  function  properly.  As  we  will  see,  under 
conditions  of  steep  terrain,  the  two  images  can  differ  sig¬ 
nificantly.  If  there  are  points  in  one  image  that  are  not 
visible  in  the  other  image,  the  results  of  the  matching  are, 
in  general,  unpredictable.  The  human  operator,  on  the  other 
hand,  can  recognize  objects  and  shapes  even  if  they  differ 
in  scale  or  aspect.  Most  present  day  automatic  compilers 
incorporate  terrain  slope  correction  features  whicn  allow 
for  on-line  geometric  correction  of  the  distortion  produced 


by  terrain  relief.  This  added  capability,  however,  reduces 
the  overall  system  speed. 
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The  automatic  compiler  is  also  less  reliable  in  re¬ 
gions  of  imagery  devoid  of  contrast,  particularly  in  tne 
presence  of  noise  which  tends  to  increase  the  difference  be¬ 
tween  conjugate  regions.  Regardless  of  the  sources  of  image 
differences,  the  machi ne  'occasi ona 1 ly  "gets  lost".  That  is, 
it  will  come  to  a  point  where  it  is  unable  to  locate  identi¬ 
cal  corresponding  imagery.  When  this  occurs,  the  machine 
will  search  until  it  is  far  off  track.  Finally  an  alarm  is 
sounded  to  notify  an  operator  of  its  condition.  Because  of 
the  occurence  of  these  conditions,  the  systems  are  only  semi¬ 
automatic  at  best. 

S te reo- comp i 1  a t i o n  is  accomplished  by  cross¬ 
correlation.  The  pictorial  information  is  first  converted 
into  electrical  signals  via  a  scanning  operation.  The  re¬ 
sulting  signals  (either  analog  or  digital)  are  then  com¬ 
pared  in  a  correlator  circuit  or  computer.  The  translation, 
or  parallax,  between  the  images  is  determined  by  the  loca¬ 
tion  of  the  peak  of  the  cross-correlation  function  indicat¬ 
ing  maximum  similarity  between  image  segments.  Performance 
for  image  registration  is  measured  by  the  accuracy  with 
which  tne  offset  of  the  peak  of  the  correlation  function 
matches  the  true  parallax  created  by  topographic  relief  and 


sensor  ori  entati  on . 
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The  term  "correlation"  is  a  generic  term  that  in¬ 
cludes  a  number  of  different  algorithms  and  techniques 
which  will  be  described  in  Chapter  2.  Each  of  the  tech¬ 
niques,  however,  generates  a  correlation  function.  An 
example  of  such  a  function,  c(^),  is  shown  in  Figure  1-1  a. 
Without  loss  of  generality,  it  is  assumed  that  the  correct 
peak  position  is  at  the  origin  of  the  £  -  a  x  i  s  .  Due  to  a 
variety  of  noise  sources  to  be  described  in  Chapter  4,  there 
are  statistical  fluctuations  in  the  measured  correlation 
function,  c(£),  which  are  indicated  by  the  dashed  lines  in 
Figure  1-1.  These  fluctuations  can  produce  two  distinctly 
different  types  of  correlation  errors.  An  error  occurring 
even  though  the  correct  lobe  of  the  correlation  function  is 
chosen  is  called  a  "local  registration  error"  as  shown  in 
Figure  1-lb.  An  error  resulting  from  selecting  an  incorrect 
lobe  is  commonly  termed  a  "false  acquisition  error"  as  shown 
in  Figure  1-lc.  The  conditions  under  which  these  errors 
occur  depend  in  part  on  the  particular  correlation  tech¬ 
nique.  It  is  generally  the  case,  however,  that  local  regi¬ 
stration  errors  occur  in  regions  producing  broad  correlation 
functions  while  false  acquisition  errors  occur  in  regions 
producing  relatively  narrow  correlation  functions. 

No  matter  what  the  actual  physics  of  tne  correlation 
process,  the  correlation  calculations  are  subject  to  a  cer¬ 
tain  amount  of  error,  and  the  errors  result  in  inaccurate 
determination  of  parallax  values.  Frequently  these  errors 


Correlation  Errors 

( a )  no  error 

(b)  local  registration  error 

(c)  false  acquisition  error 
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are  catastrophic  and  lead  to  terrain  elevation  values  that 
are  impossible  or  inconsistent.  Even  when  the  errors  are 
not  ca ta s t roph i c ,  it  is  of  the  greatest  practical  utility  to 
know  the  magnitude  of  errors  that  are  associated  with  a 
particular  region  of  an  image.  The  degree  to  wnicn  a  par¬ 
ticular  region  is  "error  prone"  depends  or.  the  physical 
properties  of  the  terrain,  the  correlation  technique,  the 
implementation  of  the  technique,  and  the  properties  of  the 
image  function  as  well  as  the  various  contaminating  noise 
sources.  Knowledge  of  these  relationships  would  allow  the 
development  of  s te reo- comp i 1  a t i on  p re - p r o ce s s o r s  to  detect 
image  regions  that  are  defective  (error  prone)  in  the  sense 
that  tne  probability  of  correct  conjugate  acquisition  is  in¬ 
adequate.  With  such  a  "defect  map",  the  correlation  proces¬ 
sor  could  operate  adaptively  within  tne  ^ol lowing  options: 

(a)  Reject  the  region  and  indicate  the  need 
for  new  data  acquisition, 

(b)  Reject  the  region  and  indicate  that  tne 
region  can  oe  successfully  processed  by 
human  operators  , 

(c)  Switch  to  a  different  correlation  tech¬ 
nique  known  to  be  less  sensitive  to  the 
particular  defect, 

(d)  Enhance  the  region  in  some  way  to  improve 
tne  probability  of  correct  conjugate  ac¬ 
quis  i  t  i  o  n  , 


(e)  Accept  the  region  as  having  sufficient 


quality  to  process  as  usual. 

The  ability  to  pre-process  stereo  inages  in  this 
manner  would  allow  improved  resource  allocation  (human  vs. 
machine)  and  thus  improve  the  overall  throughput  rate  and 
economy  of  the  operation. 

It  is  the  primary  purpose  of  this  research  effort 
to  examine  the  behavior  of  digital  image  matching  techniques 
in  terms  of  the  relationship  between  measurable  image  proper¬ 
ties  and  known  algorithm  c ha r ac t er i s t i cs  and  to  develop  image 
ouality  measures  wnich  can  be  employed  to  predict  the  per¬ 
formance  of  automatic  stereo-compilation  equipment.  In  the 
remainder  of  tnis  chapter  we  briefly  review  the  basic  con¬ 
cepts  involved  in  stereo-compilation. 

Stereo-Compilation  Concepts 

Figure  1-2  shows  the  geometry  assumed  in  the  analy¬ 
sis.  To  simolify  notation,  the  functional  relationships  are 
defined  in  one  dimension  only.  Extension  to  two  dimensions 
is  direct  and  does  not  alter  any  of  the  analysis.  On  the 
ground  tne^e  are  two  quantities  of  interest:  the  elevation, 
etX),  and  a  corresponding  intensity  pattern,  I(X),  which  is 
the  result  of  light  reflected  from  the  features  of  the  land. 
The  camera  is  located  at  an  altitude  of  H  above  reference 
elevation  and  the  focal  plane  of  the  camera  is  assumed  to  be 
parallel  to  the  reference  plane.  The  camera  has  focal  length 
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i  and  the  translation  between  images  is  the  air  base  3  .  Two 
coordinate  variables,  x  ^  and  x ^ .  are  ascribed  to  the  two 
image  planes.  The  geometry  in  Figure  1-2  is  a  so-called 
"geometric  positive";  the  true  image  lies  behind  the  focal 
point  of  the  camera  and  an  inverted  (geometric  negative) 
image  is  formed.  By  reflecting  the  image  through  the  focal 
point  a  geometric  positive  lying  in  front  of  the  camera  is 
constructed  and  has  the  advantage  of  having  the  same  sense 
of  coordinate  algebraic  signs  as  the  ground  coordinate  sys¬ 
tem  . 

In  practice,  the  acquisition  of  stereo-photographs 
rarely  results  in  images  which  are  precisely  vertical.  The 
rectification  processes  which  correct  for  differences  in 
altitude  and  orientation  of  the  sensor  stations  are  the  do¬ 
main  of  photogramme  try  (Thompson  1  955  ]  and  no  further  con¬ 
sideration  is  given  them  here. 

It  is  assumed,  then,  that  the  following  requirements 
are  fulfilled  in  order  to  obtain  "perfect"  photographs. 

(a)  The  optical  axis  of  the  camera  lens  is 
exactly  vertical  at  the  instant  of 
exposure. 

(b)  There  is  no  forward  movement  of  the 
aircraft  relative  to  the  ground  djring 
the  exposure  time. 

(c)  The  camera  is  free  of  distortion. 

(d)  Atmospneric  conditions  are  ideal. 
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It  is  obvious  that  there  is  no  practical  possibility  of  reg¬ 
ularly  meeting  these  requirements.  Fortunately,  photogram- 
metrists  have  developed  corrective  procedures  which  relegate 
these  effects  as  minor  when  compared  to  the  contaminating 
effects  discussed  in  Chapter  £. 

Given  the  geometry  of  Figure  1-2,  the  coordinates  x-j 
and  x  2  of  the  two  images  of  a  single  ground  point  located  at 
a  distance  X  from  the  nadir  point  of  image  1  can  be  deter¬ 
mined  by  applying  the  rules  of  similar  triangles.  From 
Figure  1-2  it  is  clear  that 

(1-D 

The  accumulation  of  image  points  forms  the  intensity  pat¬ 
terns  observed  in  tne  image  planes.  The  two  intensity  (or 
density)  patterns,  g^(x^)  and  g^x^),  correspond  to  the  pro¬ 
jection  of  tne  intensity  pattern  I ( X )  into  the  two  image 
planes.  Assuming  that  the  reflectance  of  the  object  field 
is  independent  of  the  camera  station,  the  procedure  for  con¬ 
structing  an  intensity  pattern  in  the  image  from  an  inten¬ 
sity  pattern  on  the  ground  can  be  characterized  by  the  ex- 
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Equation  (1-1)  can  also  be  solved  for  e(X): 


e  ( X ) 


(1-3) 


which  shows  that  the  elevation  information  is  encoded  in 
Equation  (1-1)  in  the  form  of  the  difference  in  coordinates 
(parallax)  of  the  conjugate  points  x-j  and  x^ .  The  goal  of 
stereo-compilation  systems,  then,  is  to  accurately  identify 
conjugate  points. 

The  effects  of  a  parallax  error  on  the  computed 
ground  coordinates  is  shown  in  Figure  1-3.  In  a  digital 
system,  the  images  are  sampled  to  produce  discrete  resolu¬ 
tion  elements  (pixels).  Assuming  such  a  system,  we  substi¬ 
tute  for  the  parallax  in  (1-3),  x ^  -  x  2  ,  the  parallax  in 
terms  of  the  number  of  pixels 

x i  -  x  2  =  idx  -  j  Ax  =  (i  -  j  )  A  x  (1-4) 

where  Ax  is  the  sampling  distance.  From  (1-3), 


e(X)  =  H  - 


B  i 


0-5) 


(i  -  j ) Ax  ’ 

If  the  parallax  is  in  error  by  e  pixels,  the  elevation  er¬ 
ror  is  given  by 


-  B-ES  /  1  r  \ 

06  =  [(i  -  j)  -  e](i  -  j ) Ax  ■  (1'6) 

It  is  usually  the  case  that  the  parallax  errors  are  small 
with  respect  to  the  actual  magnitude  of  the  parallax,  i.e., 
(i  -  j) >>e .  Thus  (1-5)  and  (1-6)  can  be  combined  to  give 
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While  the  parameters  H ,  B,  and  l  can  be  chosen  during  the 
data  acquisition  stage,  the  only  parameter  available  to  the 
processor  is  i  x ,  the  sampling  interval.  Some  image  regis¬ 
tration  algorithms,  however,  attempt  to  improve  on  this  res¬ 
olution  limit  by  interpolating  the  sampled  correlation 
function  in  the  vicinity  of  the  peak  [Pearson,  et  al.  1977]. 
This  procedure  is  generally  avoided  in  digital  stereo¬ 
compilation  systems  due  to  the  increased  computational  load. 
Correlation  errors  also  result  in  ground  position  errors 
denoted  by  i X  in  Figure  1-3.  Applying  similar  triangles  to 
the  geometry  of  Figure  1-3  reveals  that 


Because  of  the  two  dimensional  terrain  error,  the  perform¬ 
ance  of  stereo-compilation  systems  is  expressed  in  terms  of 
the  parallax  error. 

Most  of  the  s te reo- comp i 1  a t i on  systems  currently  in 
use  were  designed  primarily  for  producing  contour  and  pro¬ 
file  outputs.  Recently,  however,  there  nas  been  an  increas¬ 


ing  need  for  topographic  data  in  the  form  of  digital  gria- 
point  elevations.  Although  many  of  the  existing  systems  can 
be  used  to  obtain  such  data,  the  AS-11B-X  stereo-mapper  was 
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developed  specifically  for  this  purpose  [Brumm,  et  al.  1976 
As  a  result  of  improved  image  scanning  techniques  and  highly 
parallel  high  speed  digital  processes,  the  AS-11B-X  is  cap¬ 
able  of  compilation  speeds  on  the  order  of  10  to  50  times 
faster  than  for  conventional  systems  with  comparable  accu¬ 
racy.  One  of  the  features  of  the  AS-113-X  which  contributes 
to  the  increase  in  throughput  rate  is  that  image- intensity 
(or  density)  is  scanned  and  measured  on  each  photograph 
along  corresponding  epipolar  lines.  Epi polar  lines  are  the 
lines  of  intersection  between  the  photographs  and  the  set  of 
epipolar  planes  defined  by  the  focal  points  of  the  two 
camera  stations  as  shewn  in  Figure  1-4.  With  such  a  scan¬ 
ning  technique,  the  correlation  function  need  only  be  one- 
dimensional  since  conjugate  points  lie  on  corresponding 
epipolar  lines.  Any  residual  parallax  in  the  direction 
perpendicular  to  the  epipolar  lines  is  considered  to  be 
negligible  [Brumm,  et  al.  1976]. 

Thesis  Outline 

In  Chapter  2,  we  present  a  number  of  correlation 
techniques  that  are  commonly  employed  in  image  matching 
aoplications  and  indicate  tne  theoretical  justification  for 
tneir  use.  It  is  shown  that  the  maximum  likelihood  similar- 
i ty  measure  takes  the  form  of  covariance  (matched  filter), 
normalized  covariance,  or  least  squares,  depending  on  the 
a-priori  assumptions  about  the  image  formation  model. 


figure  1-4.  Epi polar  Scan  Lines 
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The  Cramer-Rao  lower  bound  on  the  accuracy  of  any 
unbiased  estimator  has  been  derived  [Van  Trees  1968]  in  the 
context  of  radar  target  range  determination  by  estimation  of 
signal  round-trip  time  delay.  A  similar  bound  has  been  de¬ 
veloped  [Knapp  and  Carter  1976]  in  the  context  of  estimat¬ 
ing  time  delay  between  received  signals  at  two  spatially 
separated  sensors  used  in  passive  sonar  systems.  In  Chap¬ 
ter  3,  these  results  are  presented  and  compared  and  their 
application  to  tne  prediction  of  errors  in  stereo-compilation 
is  discussed.  The  form  of  the  bound  derived  by  Knapp  and 
Carter  [1976]  is  shown  to  apply  to  the  image  formation  model 
assumed  and  is  thus  considered  as  an  image  quality  measure. 

It  is  impossible,  except  under  severely  restrictive 
conditions,  to  derive  a  general  expression  for  the  probabil¬ 
ity  of  a  false  acquisition.  Attempting  such  a  derivation, 
however,  leads  to  an  image  quality  measure  which  is  similar 
to  the  "figure  of  merit"  ascribed  to  correlator  performance 
by  Webber  and  Delashmit  [1974a].  This  figure  of  merit  is 
shown  to  be  a  function  of  both  image  area  and  local  signal- 
to-noise  ratio. 

In  order  to  employ  the  figure  of  merit  as  an  image 
quality  measure,  it  is  necessary  to  determine  the  correlator 
output  s i gna 1  - to-noi se  ratio.  In  Chapter  4,  the  covariance 


function  is  analyzed  in  detail  to  determine  the  degree  to 
which  sensor  noise,  quantization  noise,  "self"  noise,  and 
machine  noise  corrupt  correlator  performance.  Models  for 
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computer  round-off  errors  are  developed  for  both  fixed  point 
and  floating  point  correlation  processors.  The  expressions 
developed  for  the  purpose  of  predicting  correlator  perform¬ 
ance  all  require  the  assumption  of  input  data  (image)  sta¬ 
tion  a  ri  ty  .  Chapter  4  concludes  with  a  discussion  of  the 
effects  of  input  data  non-stationarity  on  correlator  perform¬ 
ance.  Two  schemes  for  detecting  such  regions  are  presented 
and  the  conditions  under  which  such  regions  are  considered 
"defective"  are  discussed.  It  is  shown  that  the  covariance 
processor  is  particularly  susceptible  to  data  non-stationarity. 

In  Chapter  5  we  consider  the  effects  of  terrain  vari¬ 
ability  on  correlator  performance  and  develop  a  model  for 
"relief  distortion"  which  is  described  by  the  function  re¬ 
quired  to  map  an  image  region  onto  its  conjugate.  Under 
simplifying  assumptions,  this  mapping  is  shown  to  be  a  spa¬ 
tial  contraction  or  expansion  in  the  direction  parallel  to 
epi polar  lines.  The  distortion  modelled  by  this  affine 
transformation  has  been  analyzed  in  detail  by  Mostafavi  and 
Smith  [1973a,  1973b].  We  review  these  results  and  discuss 
the  conditions  under  which  they  apply  to  the  stereo¬ 
compilation  problem.  Of  particular  interest  here,  however, 
is  the  degree  to  which  the  effects  of  distortion  on  corre¬ 
lation  accuracy  can  be  predicted.  To  this  end,  an  "image 
overlay  quality"  measure  is  developed  in  order  to  predict 
the  statistics  of  a  parameter  used  to  describe  distortion 
within  a  region  of  imagery.  Due  to  the  assumption  required 
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to  make  the  problem  tractable  and  the  computational  com- 
plexity  of  the  resulting  procedure,  it  is  concluded  that 
pre-processing  alone  is  unlikely  to  provide  all  the  needed 
information  for  accurate  error  prediction  but  that  a  com¬ 
bination  of  p r e- p r oce s s i ng  and  on-line  (i.e.,  during  stereo¬ 
compilation)  algorithms  has  the  potential  for  detecting 
problem  regions  and  increasing  the  overall  system  throughput 
rate. 

Chapter  6  is  concerned  with  the  design  and  implemen¬ 
tation  of  computer  algorithms  to  assess  the  utility  of  the 
image  quality  measures  developed  in  previous  sections  in 
terms  of  the  degree  to  which  image  "quality"  is  indicative 
of  the  correlation  behavior.  Here  we  look  at  the  image 
quality  measures  based  on  the  Cramer-Rao  bound  and  false 
acquisition  probabilities  as  well  as  a  number  of  other  image 
features  based  on  contrast  measures  such  as  variance  and 
contrast  modulation  which  appear  to  have  potential  for  pre¬ 
dicting  error  prone  imagery.  Results  indicate  tnat  a 
measure  based  on  the  false  acquisition  probability  appears 
to  nold  t.ne  greatest  promise  for  predicting  correlation 
behavior  althougn,  with  few  exceptions,  tne  quality  meas¬ 
ures  are  remarkably  similar  in  detection  performance. 

One  dimensional  correlation  simulations  in  the 
presence  of  relief  distortion  indicate  that  extreme  relief 
can,  indeed,  be  a  source  of  correlation  error.  Although 
some  s te r eo - comp i 1  a t i on  systems  correct  for  geometric 
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distortions  described  by  affine  transformations,  we  show 
that  computational  savings  may  be  gained  by  performing  these 
corrections  only  under  certain  conditions  which  can  be  par¬ 
tially  specified  in  a  pre-processing  mode. 

Finally,  in  Chapter  7,  the  implementation,  both 
optical  and  digital,  of  promising  p re- p roce s s i n g  algorithms 
is  considered.  Also  considered  are  image  enhancement  pro1 
cedures  which  can  be  used  to  reduce  the  effects  of  noise  on 
correlation  behavior.  Initial  experiments  indicate  that 
these  noise  suppression  operations  can  significantly  improve 
correlation  accuracy. 


CHAPTER  2 


CORRELATION  TECHNIQUES 

Numerous  correlation  techniques,  both  analog  and 
digital,  have  been  proposed  for  the  purpose  of  image  regi¬ 
stration.  None  of  these  is  equivalent  to  stereo-viewing  as 
accomplished  by  human  observers.  Owing  to  his  adaptability, 
the  human  observer  is  far  superior.  The  interest  in  develop¬ 
ing  automatic  systems,  however,  has  led  to  a  variety  of  com¬ 
putational  procedures.  Some  of  these  procedures  have  a 
sound  theoretical  basis  whereas  others  are  at  least  partly 
ad  hoc.  In  this  chapter,  we  present  the  theoretical  frame¬ 
work  for  some  of  the  more  common  digital  procedures  and 
briefly  discuss  the  rational  behind  the  development  of  the 
ad  hoc  techniques. 

Correlation  Geometry  and  Notation 

Let  two  images,  S  the  search  region  and  R  the  match 
window,  be  defined  as  in  Figure  2-1.  S  is  an  L  x  K  array 
of  digital  picture  elements  (pixels)  and  R  is  an  M  x  N 
array  such  that  M  <_  L  and  N  ^  K.  It  is  assumed  that  enough 
a-priori  information  is  known  about  the  search  and  match 
windows  so  that  L,  K,  M ,  and  N  may  be  selected  with  the 
guarantee  that,  at  registration,  a  complete  subimage  is 
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contained  within  the  search  area.  In  stereo-compilation 
systems,  there  are  two  mechanisms  for  establish, ng  such  a 
guarantee.  In  a  "non-tracking"  correlator,  the  search 
limits  can  be  set  by  a-priori  knowledge  of  the  maximum  and 
minimum  elevation  over  the  region  of  interest  as  indicated 
in  Figure  2-2.  In  a  tracking  correlator,  however,  the 
search  can  be  reduced  by  predicting  the  location  of  the 
correlation  peak  based  on  previously  computed  parallax 
values  and  computing  the  correlation  function  at  several 
positions  on  either  side  of  the  predicted  location.  Because 
of  the  increased  throughput  rate  achievable  by  reduction  in 
search  extent,  the  latter  mode  is  commonly  employed  in 
present  stereo-compilers.  As  mentioned  previously,  however, 
this  mode  has  the  disadvantage  of  being  unable,  in  general, 
u  o  recover  rrom  a  'lost"  condition  without  aid  from  an  o  p  e  r  - 
a  to  r . 

The  various  correlation  procedures  search  over  the 
allowed  range  of  reference  points  to  find  a  point  which  in¬ 
dicates  a  subimage  that  is  most  similar  to  the  given  window. 
In  some  applications  (e.g.,  s te reo- comp i 1  a t i on  with  epipolar 
scanning),  the  search  is  one- dimens i ona 1  (M  =  L  in  Figure 
2-1).  To  simplify  notation  we  will  assume  that  the  correla¬ 
tion  functions  are  one-dimensional.  The  extension  to  two- 
dimensional  searches  is  s t r a i gh f orwa r d  .  Furthermore,  we 
will  denote  the  match  window  R  by  defining  r  to  be  a  vector 
consisting  of  the  raster  scanned  elements  of  R,  that  is 


Match  Window-R 
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The  various  subimages  of  the  search  window  will  be  denoted 

k 

in  a  similar  manner  by  s  ,  k  =  1 , . . .  , T  where  the  superscript 
indicates  the  location  of  the  subimage  and  T  is  the  number 
of  allowed  reference  points  (taps'1). 

In  general,  one  cannot  expect  any  of  the  sub  images 

Jy 

s  to  be  identical  to  the  match  window  r.  Assuming  the 
ideal  image  acquisition  scenario  described  in  Chapter  1,  the 
primary  sources  of  the  difference  between  r  and  the  "correct" 
subimage,  sc,  are  film-grain  noise  (or  shot  noise/photon 
noise  in  a  photoelectric  sensor),  scanning  noise,  and  dis¬ 
tortion  due  to  terra-in  relief.  Film-grain  noise  is  a  direct 
consequence  o*  image  acquisition,  sensing  and  recording;  it 
is  a  random  quantity  injected  into  both  images.  Although 
this  type  of  noise  is  weakly  signal  dependent  [Andrews  and 
Hunt  1977] ,  it  is  common  procedure  to  characterize  its  be¬ 
havior  by  the  zero  mean  Gaussian  probability  density  func¬ 
tion  (pdf): 

There  is  a  correlator  output  associated  with  each  reference 
point  or  relative  location  k.  The  term  "tap"  historically 
refers  to  the  wire  tap  on  a  delay-line  correlator. 
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exp 
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9  9' 


(2-1) 


where  .  is  the  noise  variance.  It  is  also  assumed  that 
n 

the  noise  process  is  spatially  uncorrelated,  with  a  uniform 
("white")  power  spectrum.  Scanning  noise  will  also  be 
modelled  as  in  (2-1). 

The  effects  of  terrain  relief  are  net  so  easily 
characterized.  For  the  present,  we  assume  that  the  effects 
of  relief  are  negligible.  In  Chapter  5,  we  discuss  the  con¬ 
ditions  under  which  this  assumption  is  valid. 

If  we  let  g  be  the  underlying  image  intensity  or 
density  sequence  obtained  via  the  projection  relationship 
in  (1-2),  then  a  model  for  tne  effects  of  film-grain  noise 
(and  any  other  additive  noise)  is  given  by 

r  =  9  +  n„ 
c 


(2-2) 


s  '  =  3  +  n  s 

where  n  is  the  noise  sequence  contaminating  the  match  win¬ 
dow  and  n .  is  the  sequence  contaminating  the  c  o  r  re  c  t  s  ub  - 


image.  From  (2-2)  it  is  easily  seen  that 


r  =  s  n  -  n 
-r  -  s 


(2-3) 


If  we  lump  the  noise  terms  together,  i.e.,  replace 
nr  '  ns  ^  n<*’  ':hen  (2-2)  becomes 


r  =  s 


2-4) 
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c  c 

Before  proceeding,  we  note  that  n  and  s  are  corre¬ 
lated  through  Equation  (2-2).  Since  we  do  not  know  which  of 

c  k 

the  T  subimages  is  s  ,  each  subimage  s  must  be  tested.  In 
the  following  section,  it  is  shown  that  some  of  the  more 
common  correlation  procedures  are  maximum  likelihood  parallax 
es  tima  tes . 


Maximum  Likelihood  Correlation  Procedures 

If  we  consider  r  to  be  a  signal  obtained  at  the  re¬ 
ceiver  end  of  a  transmission  channel,  then  the  set  of  sub- 

k 

images  s  ;  can  be  viewed  as  the  set  of  possible  transmitted 

messages.  Given  r,  we  wish  to  determine  which  transmitted 

k  — 

signal  s  ,  k  =  1,...,T  most  closely  resembles  r.  The  maxi¬ 
mum  likelihood  (ML)  processor  selects  the  signal  which  maxi- 

k 

mizes  the  conditional  pdf  p ( r , s ' ) .  As  we  will  see,  the  form 

of  the  processor  depends  on  the  assumptions  made  about  the 

k  > 

set  of  signals  s  ;  as  well  as  the  properties  of  the  "trans¬ 
mission  cnannel".  In  order  to  generalize  (2-2),  we  intro¬ 
duce  the  parameters  a  and  m  in  such  a  way  that 

„  c 

r  =  a  g  +  n 

c  2  ~r 

(  2-5  ) 

sc  =  g  +  rn  +  n„ 

2  -  c  -c 

The  parameter  a  can  be  viewed  as  a  scale  factor  in¬ 
troduced  to  account  for  differences  in  image  contrast  and 
m  c  is  a  bias  /ectcr  modelling  possible  differences  in  mean 
density  level.-  we  will  assume  mc  consists  of  identical 
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elements  of 

value 

m .  Rewriting 

(2- 

5 )  as 

i  n 

(2- 

3) 

,  we  have 

r 

=  a  ( s C  -  m 
c  _  _  c 

?c} 

+  n 
_  r 

■ 

(2-6) 

The 

maximum  likelihood 

problem  can 

now 

be 

formu I  a  ted 

in  terms  of 

the  T 

hypotheses: 

H1 

:  r  =  a]  (s'  - 

ml 

-  nl} 

+ 

"r 

H2 

:  r  =  a2(s2  - 

?2 

-  n2) 

.f 

n 

.  r 

(2-7) 

:  r  =  aT(sT  - 

m_ 

.  i 

’  ?T} 

+ 

"r 

• 

Before  proceeding  with  the  determination  of  the  ML  processor, 

we  need  to  obtain  appropriate  expressions  for  a^  and  m^.  To 

k  t  h 

determine  m^,  we  denote  by  r.  and  s^  the  i  elements  of  the 
u 

vectors  r  and  s  respectively.  Taking  expectations  with  re¬ 
spect  to  these  elements  gives 

E ( r i }  =  ak(EisJ}  -  m)  .  (2-8) 

If  we  estimate  the  ensemble  means  by  computing  the 
sample  average  over  the  appropriate  region,  then 

m  =  7k  -  r  (2-9) 

ak 

and  '  2- 7  )  becomes 

H  k  :  r-r  =  ak(sk-sk-  nk)  +  np  .  (2-10) 

To  obtain  a^  we  compute 


(  r  -  r  ) 


r  -  r 


:v 


r  k  — k  \  /  k  -k  \  •  r  ,  .  /  «  .. 

\s  -s  -nk)  (s  -s  - n k) ;+■  E  ■.  n  ' n ^  ;  (2-11 


> 

> 
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where  orthogonality  eliminates  the  cross  terms  on  the  right 
hand  side.  If  the  vectors,  r,  s  ,  etcetera  consist  of  N 
elements,  then  the  expectations  in  (2-11)  can  be  estimated 
by  computing  sample  variances  over  appropriate  regions,  i.e., 


..  -  2-  2  .  -  2  .  ,  k  -k  w  ,  -  2  *  2 

Nar  'ak[Ncs  '2ElkS  _s  >  nk?+icn  !+;an 


(2-12) 


2  2  2 

where  5  ,  ?s  ,  and  3n  are  sample  variances  and  N  is  the 

total  number  of  elements  in  the  match  window.  The  remaining 
expectations  in  (2-12)  can  be  evaluated  with  the  help  of 
(  2  -  5  )  ,  i.e., 


Ei(ik)"k)*i^E{?'V"Ei”k'V+E!"k'V'fl%2 


(2-13) 


Thus  (2-12)  reduces  to 


2-2-2  -  2  , 

-  a a ,  -  c  +  a 


(2-14) 


from  which  it  follows  that 


.2  -  ar 


-2  -  2 


n 

2  -  2 

-  a  _ 


(2-15) 


For  high  quality  stereo-pair  imagery,  it  is  usually 
'  2  '  7  '2  -  ?  , 

the  case  that  -  >>  c  “  and  ?  >>  z  •  s  i  g  na  1  -  to  -  no  i  s  e 

r  n  s  n  3 

ratios  of  30d8  are  typical).  Thus, 


2  ,  ar 


(  r  -  r )  (  r  -  r ) 

uk  -  Fk > - ( 5 k  -  rk) 


(2-16) 


'V 


The  problem  of  determining  p(r's  )  is  complicated  by 

the  fact  that  the  noise  in  (2-10)  is  signal  dependent.  Useful 

k  k 

results  are  obtained,  however,  by  assuming  that  s  and  n  are 
statistically  independent  so  we  proceed  along  this  path  for 
the  oresent.  Under  this  assumption,  (2-10)  is  rewritten  as 


h  k  ••  : 


—  -  ,  k  —  k  ,  ,  k 

r  =  a  k  ( s  -  s  )  +  n 


12-17) 


where 


?  =  ?r  ‘  Vk 


(2-18) 


Since  the  only  random  quantity  in  (2-17)  is  nK  (r  and 

|/ 

s  are  known  samples  of  a  random  process),  then  it  follows  that 


p(r',sk; 


;xp[4r(r-r-ak(sk-sk)) 'C*1  (r-r-ak(sk-sk) ] 


nK  (2-19) 

2 

where  C  =  r  ,1  is  the  noise  covariance  matrix  (I  =  identity 
n  n  k 

2  2  2 

matrix)  and  from  (2-13),  a  ,  =  (1  +  a,  )  c  .  Tne  selection 

'  n  k  v  k  n 

of  the  hypothesis,  H.,  to  maximize  (2-19)  is  known  [Whalen 
1971]  ,  [Van  Trees  1  9631  to  be  the  hypothesis  which  maximizes 
the  quantity 


c k  =  2ak(r  -  r) 


'(sk  -  I*)  -  ih sk  -  skr(sk  -  t)  .  (2-20) 


_  * 

Here  (r  -  r)  (s  -  s  )  is  simply  the  cross-covariance  of  r 
with  s  and  (s  -  s  )  (s  -  s  )  is  the  energy  of  subimage  s  . 
If  we  make  the  a-priori  assumption  that  ak  =  1  for  all  k,  then 


the  ML  processor  computes  the  quantity 

cR  -  2 ( r  -  r)\sk  -  I*)  -  (sk  -  ?V(sk  -  t) 


12-21  ) 
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and  selects  tr.e  hypothes  i  s  which  m a  ;<  i i  z <-■  s  c  s  i  n  m 

k 

(r  -  r)'^r  -  r)  is  independent  of  the  hypothesis  we  coula 
just  as  well  compute 

-  ('ck  cm -CM 
W  ->  i  ->  -  s  j 


c,K  =  (r  -  r)-(r  -  r)  -  2<vr  -  r)'(sk  -  s' 


-  [ ( r  -  r)  -  (sk  -  sk)i'i(r  -  r )  -  (s 


3k)i 


(2-22) 


and  select  me  hypothesis  Hk  to  mini  ;;i  i  z  e  c^.  Thus,  the 
assumption  tr.at  ak  =  1  leads  to  the  "smallest  sum  of  squared 
differences"  orocessor  which  selects  the  hypothesis  which 
provides  the  best  least-squares  fit  to  r . 

If  the  assumption  is  also  made  that  tne  sub  image  energy 
is  independent  of  position  (or  varies  slowly  with  position) 
men  the  ML  processor  reduces  to  the  simple  covariance  func¬ 
tion 

ck  =  (r  -  r)  '(s<  -  sK)  .  (2-23) 

"or  tne  most  genera’,  case,  a  k  f  1  V  k .  Substituting 


(  2  - 1 c  )  into  [2-20)  a i v e : 


c,  = 
< 


2(r  -  r)  '(sk  -  s*)  ;■  r  -  r)  '(>  -  r) j 2 


>  -  rj  (r  -  r) 


r _  ,  m  -  . k  ;-k \  ,  2 


•  IS  -  s  ;  '  V  s  '  -  s  ' )  j  ■ 

(2-24) 

eliminating  quantities  that  are  independent  of  me  hy¬ 


pothesis  leads  to 


— ■  . ,  <  — k , 
rj  ; s  -  s  ) 


V  2  -  2  5 


; s 


3  /  I 


As  o  e £  o  r  e  ,  we  could  just  3S  well  comout: 


l 


(r  -  r  j  '  (  s  k  -  i‘k) 


T  (2-26) 


(  [  (  r  -  r  J  '  (  r  -  r  )  j  [  (  s  k  -  s  k  }  '  (  s  k  -  s  k  )  ]  }  2 

which  is  the  well  known  normalized  covariance  function  which 
selects  the  hypothesis  which  maximizes  the  correlation  coef - 
ficient  between  s  and  r. 

Because  of  the  computational  simplicity  of  (2-23), 
simple  covariance  has  become  the  workhorse  of  present  day 
stereo-compilation  equipment.  As  we  will  see  later,  however, 
violations  of  the  assumptions  leading  to  (2-23)  are  cause  for 
concern.  Normalized  covariance  is  commonly  employed  in  appli¬ 
cations  having  less  demanding  requirements  on  throughput 
rate  although  it  is  used  by  the  recently  developed  neterodyne 
optical  correlator  [3a 1  a s ubrama n i a n  1  976!.  In  the  following 
section,  the  image  matching  problem  vs  formulated  in  a  slightly 
different  manner  and  the  corresponding  ML  processor  is  subse¬ 
quently  different. 


Generalized  Correlation 


•  k  k _  1 

Since  s'  can  be  obtained  from  s  simply  by  a  one  pixel 

shift  in  sub  image  position,  the  matching  problem  can  be  re¬ 
posed  as 

=  S(i  +  0  x  ,  j  +  0  )  r  e  c  t  ( jjjj- ,  ^- )  +  \'(i,j)  (2-27) 

wners  2  and  D  are  the  relative  shifts  between  tne  maten 
x  y 

windows  9  and  tne  correct  subimage  of  S,  N ( i  ,  j )  is  the  signal 


dependent  noise  as  before,  ana 


recti 


*  < 


(2-28) 


[0  otherw-se 

With  tnis  formulation,  the  ML  processor  selects  the 

parameters  0  and  D  which  maximize  p { R ; D  ,D  ).  As  we  shall 
k  x  y  1  x  y 

see,  the  ML  estimator  can  be  realized  as  a  pair  of  image  pre¬ 
filters  followed  by  a  c r os s - c orre 1  a t i on  as  shown  in  Figure 
2-3.  A  ''ariety  of  "generalized  correlation"  techniques  can 
be  implemented  in  this  manner  with  the  choice  of  H  -j  and  r!  2 
based  on  the  optimization  of  certain  performance  criteria. 

If  h,(u,v;  =  H-jlu.v)  =  i  V  u,v  where  u  and  v  are  the  discrete 
horizontal  and  vertical  spatial  frequency  variables,  then  the 

estimate  3  ,3  is  determined  by  the  cross-covariance  function 
<  / 


,  N-l  M-l  .  , 

1  _  _  :  ..  x.jkUwtv 

NM  -n  ~  bSR(u’  V;/,N 
u=0  v=0 


(2-29) 


where 


W 


j  — 
J  N 


N 


(2-30) 


Equation  [2-2?)  is  simply  the  inverse  discrete  Fourier  trans¬ 
form  of  the  estimated  cross-power  spectrum  G<.  R  and  the  processor 
is  the  two-dimensional  analog  of  (2-23).  When  S  and  R  are 
filtered,  however,  the  cross-power  spectrum  between  the  fil¬ 
tered  outputs  is  given  by 

GAglu,v)  =  H1 (u,v)H2(u,v)GSR(u,v)  (2-31) 


and  the  generalized  correlation  between  S  and  R  is 


33 


where 


CAB^k-^  =  mm  :  :  vlu  ,v)GSR;  u  ,v)W  Wr<) 

u=0  v=0 


v  (  u  ,  v  )  =  H 1  (  u  ,  v  )  H  2  (  u  ,  v  ) 


12-32) 


(2-33) 


The  ML  estimator  for  Dx  and  has  been  derived  by  Knapp 
and  Carter  [1976]  in  the  context  of  determining  time  delay  be¬ 
tween  signals  received  at  two  spatially  separated  sensors  in 
the  presence  of  uncorrelated  noise.  To  avoid  the  problem  of 
signal  dependent  noise  encountered  previously,  Knapp  and  Carter 
compute  the  pdf  p(X[Q)  where 


X(u  ,  v)  = 


and  Q  is  the  power  spectral  density  matrix 


R(u,v) 

F  {  R  ( i  ,  j  )  } 

S(u,v) 

= 

F  {  5  ( i  ,  j  )  } 

(2-34) 


Q(k,t)  : 


RR 


(k,t) 


1G rS  (  k  ,  i) 


GRS(k,t) 


,Jss'v  k  ’  l; 


(2-35) 


The  assumption  is  made  that  the  elements  of  k ,  t )  are  un- 
correlated  Gaussian  [thus  statistically  independent)  random 
variables.  This  assumption  reauires  that  M  and  N  in  Figure 
2-1  are  large  with  respect  Dx  and  (this  is  not  generally 


the  case  in  s te r eo - comp i 1  a t i o n  systems) 
to  two  dimension,  is  given  by 


'he  result,  extended 


CAB(k,t)  =  fJM  ^  ^  ■:-yL('j,V/GSR(u,v)WN  WM 


(2-36] 


34 


v 


where 


^ML  ^ 11  ’  v  ^ 


A  _ 1 _ 

Jgsr(u’v)  i 


I^SR(u>v)i 

[1"]7^r(u,v)|  ] 


(2-37) 


and  R  is  the  coherence  function 


7SR(U>V) 


G  $  R  (  u  ,  v  J 


(2-38) 


/G$s  (  u  ,  v  )  G R R  (  u  ,  v ) 


Equation  (2-36)  can  be  interpreted  as  a  phase  estimator 


given  by 

Gco(u.v) 

exp[j9(u,v)]  =  — -  (2-39) 

I  gsr( u ,  v )  | 

multiplied  by  a  weighting  function  which  weights  the  phase 
according  to  the  strength  of  the  coherence.  To  implement  such 
a  processor,  Equations  (2-38),  (2-37),  and  (2-36)  need  to  be 
computed  for  each  match  window  location.  This  procedure  is 
thus  computationally  prohibitive  for  the  application  at  hand. 

The  correlation  methods  characterized  by  Figure  2-3  are 
ail  generalizations  of  (2-23).  Normalized  covariance  can  also 
be  generalized  in  a  similar  manner,  although  the  analytic  d  e  - 
velooment  of  the  optimal  processors  is,  in  general,  impossible. 
Pratt  [ 1 974]  describes  an  image  pre-processing  procedure  whicn 
simply  "whitens"  tne  images  before  performing  the  normalized 
covariance  operation.  A  major  consideration  with  any  of  tnese 
generalized  techniques  is  the  increase  in  computational  com¬ 
plexity.  To  alleviate  the  computational  load,  a  number  of 
ad  hoc  correlation  techniques  have  been  proposed. 
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Ad  Hoc  Co  r  r  elation  Techn  i  q  u_e  s 

The  techniques  described  below  are  included  here  pri¬ 
marily  for  completeness.  They  are  not  analyzed  further. 

Polarity  Correlation.  The  input  waveforms  are  quantized 
to  two  levels,  then  correlated  as  usual.  The  method  is  less 
reliable  than  covariance,  particularly  for  small  data  sets, 
although  it  is  far  less  demanding  computationally. 

Minimization  of  the  Sum  of  Absolute  Differences  .  With 
this  method,  averages  are  removed  as  in  covariance,  then  the 
absolute  values  of  the  differences  between  the  mater,  window 
and  subimage  are  summed.  The  position  at  which  the  sum  is  a 
minimum  is  considered  to  be  the  position  of  maximum  correla¬ 
tion.  This  method  is  also  inferior  to  covariance  (Helava  19781 
but  is  less  demanding  computationally  and  can  produce  useful 
results  as  in  the  following  procedure. 

Sequential  Similarity  Detection .  Sequential  similarity 
detection  algorithms  (SSDA's)  are  basea  on  the  fact  that  of 
the  (L  -  M  +  1 ) ( K  -  N  +  1)  reference  positions  shown  in  Fig¬ 
ure  2-1,  relatively  few  are  near  the  correlation  peak  (this 
depencs  cn  t.ne  a-priori  knowledge  concerning  the  predicted 
location)  and  therefore  only  relatively  few  require  high 
accuracy  calculations.  Furthermore,  for  a  grossly  mismatched 
windowing  pair  it  may  not  be  necessary  to  test  all  MN  data 
pairs  o e f o r e  rejecting  the  suoimage  as  a  possible  match 
13a  r  nea  i  Silverman  19721.  Threshold  algorithms  are  employed 
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in  which  d n  error  sequence  is  d  using  the  previously 

described  technique.  If  the  accumulated  error  exceeds  a 
threshold  (fixed  or  variable)  then  that  particular  search 
window  is  rejected.  Those  positions  which  pass  the  threshold 
test  are  subjected  to  closer  scrutiny,  in  some  cases  employing 
more  reliable  correlation  algorithms  to  make  final  decisions. 

Hierarchical  Similarity  Detection .  A  hierarchical  ap¬ 
proach  which  employs  the  SSDA  technique  just  described  has 
been  developed  by  Wong  and  Hall  [  1  978]  in  which  a  sequence  of 
images  is  created,  each  of  which  is  a  filtered,  subsampled 
version  of  the  previous  one.  An  SSDA  is  then  employed  on  the 
filtered  images  to  locate  probable  correlation  peaks.  Higher 
resolution  images  are  employed  at  each  stage  to  improve  cor¬ 
relation  reliability  in  the  vicinity  of  the  probable  correla¬ 
tion  peaks  and  the  process  continues  until  only  one  peak  re¬ 
mains.  This  method  is  similar  in  concept  to  the  multi  band 
analog  correlation  technique  which  separates  the  image  signal 
into  several  (usually  two }  bands.  The  smoothed  imagery  im¬ 
proves  the  "pull  in"  range  (due  to  the  broad  correlation 
function)  and  the  high  frequency  imagery  provides  the  required 
correlation  resolution  [Thompson  1966]. 

The  variety  of  correlation  techniques  makes  it  diffi¬ 
cult  to  develop  image  quality  measures  which  will  reliably 
predict  correlator  performance  for  all  the  techniques.  Since 
some  methods  are  considerably  more  accurate  than  others,  we 
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can  only  endeavor  to  tailor  the  image  quality  feature 3  to  the 
particular  correlation  procedure  chosen.  In  order  to  make 
the  problem  tractable,  we  will  restrict  our  attention  to  a 
small  number  of  techniques,  in  particular,  covariance  (2-23), 
normalized  covariance  (2-25)  or  (2-26),  and  1 eas t- squa res 
(2-21).  In  the  following  section,  the  accuracy  of  these 
correlation  schemes  is  assessed  by  computer  simulation.  The 
goal  of  uhe  remaining  chapters,  then,  will  be  to  develop  image 
quality  measures  which  will  predict  the  behavior  of  these 
model s  . 


Co_rr_e lation  Processor  Comparisons 

In  order  to  obtain  valid  comparisons  of  the  image 
correlation  techniques,  it  is  necessary  to  have  access  to 
stereo-pair  imagery  for  which  the  parallax  values  associated 
with  each  image  point  are  known  precisely,  or  equivalently, 
the  corres pond  i  ng  elevation  data  is  known  precisely.  It  is 
also  desirable  to  be  able  to  control  the  image  noise  content 
as  wel;  as  its  spatial  spectral  characteristics  (both  int°n- 
sity  and  elevation).  In  order  to  ODtain  the  latter  capability, 
it  is  necessary  to  generate  purely  simulated  imagery.  While 
we  have  taken  this  approach  in  part  of  the  experimental  results 
to  be  presented  in  Chapter  6,  the  imagery  so  generated  lacks 
realism.  The  disadvantages  of  real  imagery,  however,  are 


that  the  required  parallax  precision  is  not  readily  available 


V 


and  secondly,  res  1  imagery  i :  noise  coitr:;  r-i  tvd .  e  c  *.•  n  C 
devel ocnonts  i  n  digital  senso  r  :» i  m  j  1  a  ;  i  c  n  n  a  v e  m a c e  a  / a  i  1  a  t  1  o 


:  o ci p u  t e  r  software  able  to  p r o n  u c e 


?  a  s  e  w 


the  charactoristi 


P  n  d  to o  r d 


carerc  ;  i  '< n a  s  I  ,  J r  r  j  r  ,  and  A  i  s  o  a  j g n 
used  i  n  t  r. e  f o  1  1  o  w  i  n  j  :  i  m  u  1  a  t  i  o  n s  w a  : 
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frcn  a  digital  cat  a 
p  h  s  taken  by  a  f r a :r, . 
71.  The  data  case 


c  t-  ^  >*  yi 


created  r  r  o m  a  c t  u  a  . 

pno tog ra ? ns  .  7ne  terrain  model  was  developed 
using  tne  5  e  n  d  i  .<  A  S  -  1  13-1  analytical  s terecpl  otter  and  con¬ 
sists  of  3  large  array  of  ground  intensity  and  elevation  data 
w  h  i  c  n  correspond  to  samples  taken  at  15.75  ft.  intervals, 
using  tne  software  package  3 1 M  one  can  spec i -y  numerous  image 
creation  parameters  } e .  g .  ,  air  base,  altitude,  focal  lengtn, 
tilt,  oi;<el  soacing,  etc.,'  and  introduce  vertical  scaling 
1  M i  '< n a  i  1  ,  J n  r u in  ,  and  A1  spa ugn  13771.  "The  resulting  images 
can  ce  manipulated  'filtered,  corrupted  by  noise,  etc.}  as 
desired.  7  s  tereo -  pa i r  generated  using  SIM  is  s  h  o  w  n  in  Figure 
2-7  alone  with  the  coded  elevation  data  corresponding  to  the 
right  image  in  Figure  2  -  4  b  .  Pertinent  image  formation  para¬ 
meters  are  5  town  *  n  Table  2-1. 


ne  corbel  a 

t  i  o  n 

p  r  o  c  e  d 

ures  were 

performed 

oy  selecting 

i  r,  c  o  w  from 

tne 

right 

image  and 

a  search 

area  from  tne 

e  as  s  r  o  w n 

i  n  F 

'inures 

2  -  4  a  ,  b  . 

Since  max 

i  n  u  n  and 

minimum  elevations  are  known  in  this  p  a  r  t i c  u 1  a  r  model,  tne 
s  e  a  r  c  r  window  could  oe  selected  3  s  shown  in  Figure  2  -  2  .  This 
guarantees  tnat  the  correlator  will  not  "get  lost.'  ~he  image 


gure  2-4  . 
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lines  in  Figure  2--  a  re  c-  ;>  i  polar  lines  of  the  stereo-pair  so 
the  c  o  r  r  e  1  a t i o n  f u n c  t i o ns  need  only  be  on e - d i mens i ona 1  . 

Shown  in  Figures  2-5,  2-6,  and  2-7  are  the  computed 
parallax  c atria e  :  c  o r r e s  p  e  r ,  J i n  a  to  covariance,  least-squares, 
and  no '"’.a  1  i  p  e  :  :;var:  nco  respectively  c  a  r  match  window  sizes 
2  >  7  ;  3  lines  j  "  o  i  x  e  1  s 1  i  n  e  '  ,  2  .<  15,  and  3  x  23.  The 
g  r  3  y  1  e  v  •_  1  [  o  •'  ;  n.  t  •_  n  s  i  :  y  ,  display  e  c  in  t  n  e  s  e  figures  is  1  i  near!  y 
relate.:  to  tr,:  s  ’’f .  alue.  ~re  source  images  for  these 
results  were  re;ie  f  r  e  *  .  Finj'-e  2-5  snows  the  correct  parallax 


a  pc  re  i  -lately  >14  *  t 
s :  a.  c  i  n  .  ;  n  t  re  s  /  n  :  n 


ate 


n  e 

coded  e 1 e  v  a t 

ion  data 

s  0 

lotion  limit 

specific 

a  t 

icn  c a  t  a  spec 

i  f i e d  in 

0 

*  elevation. 

Due  to 

t  •; 

c  images,  t  h i 

s  resold 

? 

resent  c  0  m  p i 1 

■it icn  sy 

* 

t  r  e  c  ;  r  r  e  1  a  t  i 

0  1 6  C  n  - 

'■*  p  n  *•  r-  y  ,  -  - 

j  *■  ;  *  r  j  f 

;•>  r  '  '  n  r  P  P 

•  •- 

v  ;  :  y  3 ..  r  "  r  :  •: 

*  '  -  :  -  '  : 

e ; 

‘  ’i  r  '  :  ..  r  ~  2- 

-  : :  2  - 

_ 

i  n  c  I  - 

+  2  r  p,  j  0  r, 

■  -  •“>  r*  >r  .c 


n  u  ~  0  - 


t  ;  o  i  a  r  -  a  n , 
n  a  1  ,  note 


X 


\ 


a.  - 


that  m. a ny  of  t h c  e r r o 

r  i  ' 

■  till  i  u 

1  •  . 

.  .  U 

■if-:,  ;  r 

:  a  r  - 

m  a  1  i  z  d  c  o  v  a  r  i  a  r.  c  e  ;  1 

1  *l  ci 

1  o  n  g  t  r,  v  :>  n  t  o 

r 

.  i  r  r; 

•  --  ♦  V  ,  . 

o  '•  t'  -  ’  1  •  x 

matrices.  There  a  r e 

t  >i  0 

p  o  s  s  i  1 1  e  r  ;  *, a 

l ; 

f  .•  •_  . 

; 

i  t  r  . 

Since  the  c  o  r  r  e i a  t i o n 

f  u  n 

ctions  are  s 

:  t  0  t  a 

-■  .  ’  r. 

that  the  t  r u  e  p  e a k  e f 

X.  n  c* 

c  o  r  r  o  1  a  i  c  n  f 

t; 

' 

..  -  :  - 

i  .  :m,  r, 

t  w  c  adjacent  s  a re  p  1  •  ■  t 

T  h 

is  is  precis o 1 

J 

■  ■  ••  -  -•  - 

•t  .v-en 

match  window  1  oca  tier 

c  j  r 

respor.es  to  - 

r 

J  »  0  :  i  v.*  I . 

, .  <r  ,  .  4-  •  i 

c  r  •'  l  w  \ 

para!  lax  i  s  c  h  a  n  -g  i  n  g 

r  ,,  ... 

o n e  re  sol u t i c 

■i 

■a  '  m  o  n  t 

to  an o 

**  r>  '  •  ir 

It  is  not  u n  r  e  a  s  c n  s  L 1 

e  to 

e a  pec t  para!  1 

errors 

of  one 

pixel 

to  occur  in  such  a  situation.  The  major i ty  of  errors  lying 
along  the  contours  are  one-pixel  errors.  Also  note,  however, 
that  the  few  1  a  r  g  e  r  errors  that  occur  using  least-squares  and 
normal i zee  covariance  also  tenc  to  occur  at  these  locations, 
since  t n e  peak  of  the  correlation  function  is,  in  effect,  sup¬ 
pressed  .  -nother  possible  source  of  these  e r r o  r  s  could  be  the 
inters o 1  a t i c  n  on  the  date  base  requ i red  to  create  the  synt h e  t i ; 
imagery .  3  oca  use  these  errors  are  relatively  small,  they 

would  not,  in  genera!  .  increase  the  probability  of  a  ‘lost’1 
condition  in  a  t  r  a  c  < i n q  correlator.  For  this  reason,  these 
regions  a  re  treated  in  s-icseouent  a  n  a  1  y  s  i  s  as  o  e  i  n  o  'no-error  ' 
regions. 

Finally,  we  note  the  loss  in  correlator  resolution  wit- 
i  n  c  r  e  a  s  e  d  w  *  r.  d  o  w  length  a  s  seen  b  /  the  i  r  r  e  a  u  1  a  r  contours  i  n 
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to  the  mean  elevation  viewed  through  the  match  window.  This 
effect  is  most  noticeable  in  regions  with  steep  terrain  slope 

The  correlation  experiment  was  repeated  wits  noise- 
contaminated  imagery.  Signal  independent  additive  Gaussian 
white  noise  w as  introduced  into  each  image  of  the  s  t e r e o -■  p a  i  r 
resulting  in  an  overall  signal  -  to-noise  ratio  of  2  C  d  B  .  T  h  <:■ 
resulting  correlation  parallax  results  are  shown  in  Figures 
2-12,  2-13,  and  2-14.  While  the  covariance  parallax  matrices 
appear  to  be  almost  unchanged  (perhaps  because  cf  the  inf eric 
initial  behavior),  the  degradation  of  performance  for  the 
normalized  covariance  and  least  squares  processors  is  quite 
clearly  o d s e r v a b 1 e .  This  chapter  is  concluded  with  a  compar¬ 
ison  of  the  computation  complexity  of  each  cf  these  algorithms. 

Comparison  of  Comouta t i o n a  1  Efficiency 

In  order  to  properly  compare  the  correlation  algo¬ 
rithms,  it  is  necessary  to  compare  the  computation  comp! exit 
as  well  as  accuracy  since  a  traoe-off  generally  results  De- 
twee  n  tnese  characteri  sties .  Since  many  of  the  computations 
can  be  cone  in  parallel  with  special  purpose  hardware  it  is 
difficult  to  compare  algorithms  in  terms  o'  total  computa¬ 
tion  time.  However,  a  common  measure  of  computational  com¬ 
plex  icy  is  the  tccal  number  of  real  adds  and  multiplies  re¬ 
quires  r c r  a  particular  operation.  This  is  the  approach 
taken  nere .  Since  the  1  o a  s  t - s q  u  a  r  e  s  procedure  in  (2-22) 
requires  3  two-pass  procedure ,  one  to  compute  the 


r^r 'T"  7*’~*  < 

'*••  -**»•••’ ^4£3g£C5^^.^-. : 


*iy  ,.  ■’  ‘  •  •;  • 

:- "  £•;  i*s  £  ;v  *■* 


-V  -■■■*■■*  ■HWi'fcA  * »  y 


->»j; 


Figure  2-13.  Computed  Parallax-Least  Souar 
SNR  =  20  d  B .  .Mateo  Window  Si 
(a)  7x3,  (b)  15x3,  (c) 
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igure  2-1-1.  Computed  Parallax- Normal!  zed  Co  /a 
SNR  =  23  d 3 .  Match  Window  Size: 

(  a  )  7  x  3  ,  ( b )  1 5  x  3 ,  ( c )  2  3 
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of  an  add  opera  t  i  or1,  then  the  result:)  in  1  ..  :>  1  ?- 

pressed  in  terras  o f  e.  ji  valent  adds  a  >  ind  looted 
2  -  4  .  Con;  p  u  t  a  t  i  o  n 1  c  c  1  c  /.  i  tv  is  p  c  1 1  e  d  ■.  t  a 
(  with  the  p  a  r  a  m  c  t s i 
value).  Thus  ft r  T 

least-squares  a  n  d  n  y  r a  1  i  z  •  d  c  s  v  a  r  i  a  n  _  >. 
than  for  c  o  va  r  i  a  n  c  at  '1  =  10  and  varying  r,  •.-arl  j 
t  c  5  ~  f  or  N  -  1  0  0  .  C 1  e  2  r  1  y  ,  these  q  u  a  n  t  'ties  j 
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9  ,  the  c  om  p  u  t  a  t  i  c  m  a  1  c  c>  v.  p  1  e .«  i  t  j  • 


’  .  d  1  e  m  e  n  t  a  t  *  t  n  a  u  g 


Ti  ean  t  to  to  i  1  1  u  s  t  rati  v  e  .  0  n  e 


major  differences  to  tween  the  general  class  c -  i  mace 
s  t  r  a  t  i  o  u  p  r  o  b  1  e  m  s  a  n  d  those  t.  hat  a  p  p  1  y  s e  c  i  f  i  c  a  1  1  j 
com  p  ’litis  n  i  s  t  h  &  s  i  r.  e  o  f  t  h  e  mate  h  w  i  n  d  o  w  ;  i  r,  stereo 
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terns  a  small  'natch,  window  is  required  to  achieve  the  desireo 
resolution  and  the  computational  savings  of  covariance  (per¬ 
cent  a g  e  w i s e 1  i nc  r e a ses  with  decreasing  window  size.  If  tost 
is  no  object,  h i g n 1 j  parallel  processors  can  be  employee 
which  nearl  j  eliminate  t.nis  differential  in  terms  of  compu- 
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where  C  l:)  is  the  autDcc-varionct  function  of  the  process 
Thus  the  lower  bound  in  (3-11)  i  s  invers  e  1  j  p  r  op  o  r  t  i  c  r.  a  1  t 
the  curvature  of  a u toco vu r i once  function  measured  at  the 
origin. 

Equations  ( 3 - S )  and  ('-13)  can  be  put  in  a  sere  os 
ful  form  for  conipu  to  ti  on  on  sampled  i  macery .  Assuming  tha 
the  sampling  rate  is  sufficiently  high,  the  resulting  sam¬ 
pled  image  is  b a n d 1 i m i t e d  in  . £  cycles/sample  spacing  from 

9 

w hi  cm  it  follows  t  n  a  t  the  noise  variance,  ,  is  given  by 
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where,  since  the  units  are  different, 
magnitude  only.  "h  u  $  f  3-8)  oeccmes 
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i  he  i  ower  counc  in  \z-c.  .)  is  atcainao.e  on  .  y  cy  a 
called  ’efficient"  or  'minimum  v  a  r  i  a  n  c  e es  tin  a  tor  •  W  hale  n 
13  7  1]  .  One  of  t  n  e  a s  s  o  m  c  t i o n  s  1 e a  c i n g  to  (3-21;  is  negli- 
g  i  o  i  e  topouraenfe  r  e  1  i  e  *  ,  w  r.  i  c  n  ,  as  stated  earlier,  1  e  a  ;  s 
‘a  tier  ships  in  (3-7,.  For  tne  mere  general  case  c  f 
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The  Crarner-Tao  inequality  specifies  a  1  ewer  bound  or, 
the  parallax  error  variance  but  it  does  not  guarantee  that 
this  bound  can  actually  be  reached  or  even  closely  approxi¬ 
mated  .  For  this  reason,  it  becomes  important  to  study 
physically  realisable  correlation  processors  and  to  compare 
their  performance  to  the  lower  bound. 

Generalized  Correlation  Local  Error  Variance 


The  generalized  correlation  processors  described  in 
Cnapter  2  allow  an  analytical  approach  to  error  variance  d  e  - 
t  e  rm i n  a  t i o n .  Let 
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logons  to  .2-2  9]  .  The  average  output  c ;  ' 
processor  is  sr.own  i  r.  Figure  3-1  along  w  i  ■ 
correlation  functions .  Without  loss  of  g « 
‘  -  *  C .  The  actual  output,  c  '  •;  ■  ,  is  only 
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We  note  that  this  is  precisely  the  Cramer-Fao  lower  bound 
given  in  (3-5).  There  is  a  temptation  at  this  point  to  con¬ 
clude  that  the  ML  processor  defined  by  (3-27)  is  a  minimum 
variance  processor.  Before  reaching  such  a  conclusion,  how¬ 
ever,  we  must  access  the  validity  of  the  assumption  leading 
to  (3-5)  and  (3-27).  Three  assumptions  are  of  particular 
concern.  First,  the  assumption  that  3  is  large  with  respect 
to  tne  correlation  width  is  generally  not  valid  in  stereo¬ 
compilation  systems.  The  effect  of  a  small  match  window  is 
to  introduce  random  fluctuations  into  the  correlation  process 
which  are  not  accounted  for  in  (2-35)  [Helava  1973).  Secondly, 
tne  analysis  wnicn  leaas  to  (3-25)  assumes  no  false  acquisi¬ 
tion  errors.  For  tne  present,  suffice  it  to  say  that  false 
acouisitions  cause  an  increase  in  : " .  Finally,  all  of  tne 
preceding  analysis  assumes  wi de-sense  stationarity  of  both 
image  and  noise  processes.  ” h u  s  while  ( 3  -  23}  may  d  r  o  v i  :  e  an 
accurate  measure  of  system  performance  in  image  registration 
o r c  o 1 o ” s  involving  large  section,  of  imagery,  we  cannot  ex¬ 
pect  '/a  L  processor  to  achieve  the  accuracy  specified  by 
t  r.  e  lowe--  t  o  u  n  d  in  ,  3  -  5  )  . 

rcr  tne  simple  covariance  estimator,  .(f)  =  1,  and 
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the  "minimum  variance"  estimator.  Since  for  high 

o - n c i s e  ratios,  the  theoretical  accuracy  of  the  co¬ 
processor  converges  to  the  Cramer-Rac  bound,  the 
ao  bound  itself  is  a  reasonable  candidate  for  an 
a  1  i  t  measure. 

Specifically,  we  will  consider  (2-8)  as  an  image 
Since  G  (  f )  is  generally  unavailable,  we  must 
2(f)  (or  G r r ( f ) )  and  estimate  G  ( f )  so  that  { 3 -  7 ) 
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w  e  n  o  *  c  :  h  a  t  *  :  r  -  :  r  S  \ :  ,  tnere  s  houl  d  b  e  little,  if  a  .1  y  , 
difference  t  rt ■_  V L  processor  given  by  (2-36)  and 

(2-37)  and  si  ".pie  covariance.  Referring  to  Figures  2-3 ,  2-10, 
and  2-11,  we  see  test  both  least-squares  and  normalized  covar¬ 
iance  result  in  a  1  owe r-mea n- s qua  re  error.  Thus,  covariance 
is  oefinitely  net  a  ::  i  n  i  mum  variance  estimator.  As  discussed 
in  Chapter  4 ,  the  violation  of  local  image  stationari ty  is 
the  pi  rimary  reason  for  this  discrepancy. 

C  0  r  r  g  1  a  t  i_c  n  False  Acquisition 

As  described  in  Chapter  1,  a  false  acquisition  occurs 
when  the  peak  of  the  correlation  function  is  associated  with 
an  incorrect  lobe  (Figure  1  - 1 c )  .  For  an  arbitrary  correla¬ 
tion  processor  ■:(■:),  the  probability  that  suen  a  peak  occurs 
at  location  *  f  3  is  expressed  as 

pfa  ■(  ■)  =  ?roo  -c(  ■  ;  >  c ( 0 )]  (3-32) 
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Webber  and  Cel  ashm*'  t  ’  ’  9  7  d  a  ;  t  c  c  e  s  c  r  i  b  e  correlator  p  e  r  f  o  r  - 
mance  for  Gaussian  r  o r,  d c s  c o n >:  :  . 

In  Chapter'  J ,  we  w i  1  se  e  that  :  ;  C )  depends  on 
additive  noise  as  well  as  "  s  e  I  f  n  o  i  s  o  1  a  n  j  ' a  c  h  i  n  ;■  r,  s  i  s  e " 
which  are  error  sources  to  be  discussed  in  the  folio w i n g 
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(  2  )  Sg]f  no;;  e  result'!  ng  fro  ;:i  cno  f  •  c  t  that 
each  correlation  function  it  a  sample 
from  a  random  process  with  associate d 
statistical  fluctuations. 

(3)  Machine  noise  resulting  from  computer 
truncation  or  rcjnd-off  due  to  limited 
m  a  c  n  i  r,  e  r  e  g  i  s  t  e  r  lemons. 

( 4 )  Incut  data  non  -  $  t  at i onarl ty  which  vio¬ 
lates  the  assumption  leading  to  co- 
variance  being  equivalent  to  the  ML 
proces  sor  . 
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[Mostafavi  and  Smith  1973b].  In  general,  the  computation 
of  P  ^  involves  a  2 L  +  1-fold  integration  over  the  joint  pdf 
of  the  correlator  tap  outputs.  For  even  the  smallest  values 
of  L,  this  is  computationally  prohibitive.  Because  of  the 
correlation  between  tap  outputs,  the  only  situations  that 
can  be  handled  analytically  occur  when  the  input  data  con¬ 
sists  of  either  very  smooth  1  ov/pa  s  s  signals  or  wi  de-band  widt 
signals  approaching  white  noise.  In  the  former  case,  we  can 
assume  that  the  tap  outputs  are  so  highly  correlated  that 
the  self  noise  is  negligible  and  in  the  latter,  we  assume 
the  tap  outputs  are  statistically  independent.  Wernecke 
[1973]  presents  an  analysis  of  multitap  correlator  self¬ 
noise  for  analog  signals  in  which  the  tap  covariances  are 
accounted  for  by  performing  a  1 ea s t  -  s q ua r e s  parabolic  fit 
to  the  average  correlator  output  in  the  vicinity  of  the  cor¬ 
relation  peak.  "he  results  obtained  will  be  presented  in 
the  sections  to  follow.  They  do  not  differ  substantially, 
however,  from  t n e  results  associated  with  the  Cramer-Rao 
lower  bound  wnicn,  we  recall,  is  related  to  local  registra¬ 
tion  error  and  neglects  false  acquisition  errors. 
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Sensor  ') o  i  s e 


Let  each  image  be  corrupt 
Gaussian  uncorrelated  noise.  The  covariance  calculation  in 
the  presence  of  such  noise  becomes 

N 


c  (  k  )  =  I  (  a  .  -r  q  .  +  n  •  ) 

i  y  l  +  k  1 

i  =  1 


M  i'J 

•  (5(  *  :i>  <V 

1=1  1=1 


(4-4) 


.  +  n  .  ) 
k  i  ' 


where  c  is  an  estimate  of  the  true  discrete  covariance  func¬ 
tion 
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anc  ;  •;  •  and  in;  are  statistically  independent  white  noise 
sequences.  The  compu ta t  i  ona 1  error  due  to  the  additive  nois 
is  given  by 
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Taking  expectations  with  respect  to  the  noise  terms 
straightforward  to  show  that 
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If  we  also  assume  stationer ity  of  the  input  process 
1,  expectations  in  ( 4  -  7 )  can  also  be  taken  with  respect 
process  g  resulting  in 


.2  , 


2  „  2  2 
-  -  +  T 


■  \  k )  -  M :  1 2  z  ■  j  i 

e  .  n  g  n 


(4-9) 


we  let 


f(:  g2  .  1  n  )2)  ;  Nj2 

-i  3i  M  '  "  =  i  '  '  g 


(4-10) 


r  t  -  n 

v  "  i  M 


2  1  <2,  _  ,2 

'  -  ^  I  ~  4  - 


n 


(4-11) 


Since  t n e  error  in  (4-5)  usually  involves  a  large 
0 f  terms,  it  is  reasonacle  to  assume  tnat  tr 
is  Gaussian. 
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Self  •  1  o  i  s  e 


If  the  image  input  signals  consisted  solely  of  tne 
desired  signals  (  i  .  e .  ,  g  .  and  g  .  +  ^ )  there  'would  still  be 
statistical  fluctuations  at  the  correlator  output  due  to 
the  random  nature  of  the  desired  signals.  These  fluctuations 
are  known  as  correlator  "self-noise". 

Taking  expectations  in  (4-1), 
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where  R^s  is  the  cross-correlation  function 
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and  stationarity  is  assumed.  The  cross-correlation  be  twee i 
correlator  taps  is  given  by 
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In  general,  the  fourth  order  moments_  in  (4-14)  cannot 
be  simplified.  However,  if  we  assume  that  the  error  incurred 
by  estimating  tne  means  in  (4-1)  is  negligible  then  the  co- 
variance  function  can  be  written  as 
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r  and  s  are  zero  - "lean  sequences.  With  t.nis  a s s u m d - 
only  the  first  term  in  (4-14)  remains. 
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Thus  the  cross-covariance  between  correlator  taps  is 


given  by 
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Since  g  is  zero-mean,  the  fourth  order  moment  in  (4-20)  can 
be  simplified  if  we  assume  the  process  generating  g  is  Gaus¬ 
sian.  In  this  case  [Whalen  1971,  p  .  9  7 ]  , 
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and  (4-20)  becomes 
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Note  era:  the  last  term  in  (4-23)  is  the  contribution  to  the 
tap  /ariance  due  to  tne  additive  sensor  noise.  Thus  tne 
additive  noise  and  the  self-noise  o  *  the  desired  signal  are 
uncorrelated.  If  g  is  a  white  process  then  (4-22)  becomes 
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(4-25; 


The  contribution  of  self- noise  to  the  degradation  of 
correlation  accuracy  is,  in  general,  difficult  to  deter -nine 
due  to  the  correlation  between  correlator  tap  outputs  cv/en 
by  (4-22).  As  mentioned  earlier,  '/Jernecke  [1978]  presents 
an  analysis  in  which  the  tap  covariances  are  accounted  for  by 
performing  a  1  e?  s  t- s  qua  re  s  parabolic  fit  to  the  average- 
correlator  output  in  ^he  vicinity  of  the  correlation  peak. 
With  the  a  s  s  u  t p  t i o  n  s  that: 

(1)  The  correlator  taps  are  centered  on  the 
correct  peak  location, 

(2)  The  variance  of  the  correlation  function 
curvature  is  negligibly  small,  and 

(3)  Tne  correct  correlation  lobe  is  acquired, 
Wernecke  concludes  that  the  parallax  error  variance  is 
given  by 
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where  3  is  defined  in  (3-14)  and  the  total  number  of  corre¬ 
lator  taps  is  2 L -r  1  .  Note  that  assumption  (3)  above  ignores 
false  acquisition  errors.  If  the  correlator  tap  outcuss  are 
statistically  ’naeoendent  then  (4-25)  becomes 
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Comparing  this  result  with  (4-3)  we  see  that  under  the  above 
assumptions , 
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2 

The  fact  that  r  in  (4-27)  is  inversely  proportional 

to  L°  is  somewhat  misleading  since  the  addition  of  correlator 
2 

taps  reduces  3  only  as  long  as  the  additional  correlator 
taos  improve  the  parabolic  curve  fitting  technique.  For 
severely  bandlimited  signals  resulting  in  broad  correlation 
functions,  such  a  result  may  be  appropriate.  However,  in  the 
case  of  severely  bandlimited  signals,  it  is  highly  unlikely 
that  tne  correlator  taps  are  statistically  independent.  To 
avoid  tnese  problems,  one  must  use  (4-25)  and  constrain  L 
to  a  reason  ably  small  value.  The  computation  of  (4-26)  on 
match  window  sice  subsections  is  cumbersome  due  to  the  cal¬ 
culation  of  the  covariance  matrix  '<  .  Equation  (4-25)  is 
thus  not  well  suited  to  be  used  as  an  image  quality  measure 


or  pre-processor  algorithm. 
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M  a  c  h  i  n  e_  Noise 

Figure  4-1  shows  a  block  diagram  or  the  computation 
of  the  covariance  function  as  performed  by  trie  AS-113-X 
s  t  e  r  e  o  -  c  c  :n  p  i  1  e  r  .  The  computation  of  c  ^  is  subject  to  errors 
due  to  truncation  or  round-off  in  the  limited  precision  corn- 
outer.  The  magnitude  of  the  errors  incurred  depend  on  the 
number  of  binary  digits  (bits)  used  to  represent  the  input 
data,  the  number  of  bits  used  in  the  arithmetic  operations 
and  the  type  of  arithmetic  employed.  Most  present  stereo¬ 
compilers  employ  fixed-point  arithmetic  with  3  bit  precision. 
Accuracy  in  fixed  point  correlators  suffers  from  the  need  to 
scale  the  input  data  in  such  a  way  as  to  avoid  register  over¬ 
flow  in  the  accumulators.  Vlith  continuing  improvements  in 
m i c r o p r o c e s s o r  and  minicomputer  technology,  we  will  see  the 
advent  of  15  oit  stereo-compilers  in  the  very  near  future. 
r  1  o  a-t  i  n  g  ooint  orocessors  eliminate  the  need  for  prescaling 
and  i mo  rove  overall  system  precision  at  the  expense  of  in¬ 
creased  system  complexity  and  cost  and  decreased  throughput 
rates.  Developments  in  tne  digital  processing  of  radar  and 


sonar  signals,  however,  have  led  to  implementations  o 
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In  this  section, 


of  both  fixed-point  and 


an  analysis 

floating-point  covariance  processors  is  presented  and  expres¬ 
sions  far  the  contribution  to  correlator  output  noise  due  tc 
finite  register  lengths  are  derived.  We  will  consider  the 
"correct1'  computation  to  be 


with  the  actual  computed  function  given  by 


iSi  +  k  +  e!>  -  5f  ri  +  e2)(:  si  +  k  + 


(4-3C) 

where  e ^ ,  e ^ ,  and  e  ^  are  the  resulting  accumulator  output 
errors  and  e ,  is  the  result  of  division  by  N  and  subtraction. 
It  will  be  assumed  that  the  distributions  of  e^  and  are 
identical . 


Fixed  J  o i n  t  Processing 

For  fixed-point  arithmetic,  one  usually  cons  i  a 

register  to  be  a  f i xed-poi nt  fraction.  In  this  way  toe  pro¬ 
duct  o  r  two  numbers  remains  a  fraction  which  must  be  trun¬ 
cates  or  rounded  off  to  remain  within  tne  available  resisted 
length  o -  b  Dios.  The  sum  of  two  numbers,  however,  dees  no; 
reauire  truncation  although  it  can  produce  an  overflow  condi¬ 
tion,  i.e.,  the  sum  is  no  longer  a  fraction.  Inis  overflow 
concition  i  ~:poses  a  constraint  on  tne  dynamic  range  of  the 
correlator  output  w n i c n ,  in  turn,  requires  scaling  of  the 
input  data.  More  soeci f i c  a  11 y  ,  if  we  assume  t  n  a  t 


V 


U  i  r.  £  i 

°  i  -  1 

for  all  values  of  i  and  k,  then  it  is  necessary 

N 

i  =  1 
M 


(4-31 


.  d  t 


ri  s1+k 


1 


i  =1 


r  . 


Si  +  k 


1 


<  1 


Vk 


where  tne  prime  notation  denotes  pre-scaled  quantities. 
T'nis  condition  can  be  achieved  by  computing 


c. 


N 

i  =  1 


r  a  S  i  -  k 


N  r  i 

i  =  1  ' 1  i  =  1 


s  • 


i  +  k 


(4-33) 


~he  comouted  covariance  function  is  thus  given  by 


c. 


N  r  .  s  .  ,  , 
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i  =1 
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-  *  -y 
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the  quantization  error  is  additive,  the  output  error  due  to 
quantization  has  precisely  the  same  form  as  the  output  error 
due  to  sensor  noise.  It  follows  from  (4-9)  that 


2-1  2  . ,  2  2 

cr  =  vr  a  f  Za  „  +  a. 

cq  9  g  q 


(4-37) 


where  a  is  the  output  variance  due  to  input  quantization 

cq 

2 

noise  having  variance  a  .  For  quantization  to  b  bits,  it 
can  be  easily  shown  that  (Oppenheim  and  Schafer  1975] 


2  _  2' 


(4-38) 


The  factor  of  N ^  in  (4-37)  (when  compared  to  (4-9))  is  the 
result  of  division  by  in  (4-35).  From  (4-33),  (4-34), 
and  (4-35)  it  follows  that  the  error  in  the  partial  sum  in¬ 
jected  by  round-off  and  scaling  is  given  by 


N1  r.  r.  N1 

l  (II  +  0  )  .  /_L  _i  r 

i--l  X  ^2  i  =  l  N1  i=l 


'’1  s.  ”1 

0.  +  z  Jj-  z  n . 
1  i  =1  H  i  =  1 


+  Z  r(i  Z  o,  +  3} 

1  ■ 1  1=1 


(4-39) 


where  3  is  the  error  incurred  with  division  by  For 

round-off  errors,  E { e  >  =  0.  For  truncation  errors,  there 


is  a  bias  term  since  truncation  always  rounds  down  when  the 
numbers  to  be  truncated  are  positive.  We  will  ignore  the 
distinction  here  and  consider  only  round-off  errors.  Since 
{-},  {a},  in),  and  {3}  are  statistically  independent  and, 
we  assume,  uncorrelated  with  {s}  and  { r } ,  it  follows  that 


E ( e  2 )  =  cj 
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N1  5-  2  N1  2  N1  r.  N1  2 

*  E(  E  jp)  +  E(  z  0.)  +  -L[E(  z  -1  z  5.) 

isl  N1  1=1  1  h\  1=1  N1  1=1  1 


N1  s.  N1  2 
+  E(  Z  B-  Z  n  •  ) 
1=1  N1  1=1  1 


+  E( 


N1  N1  N1  2 


ni  5i  .Z. 


i=l  '  1=1  1  1=1 


^  )  J  +  E(s‘) 


fT~  aq  +  N1aq  +  \  aq  1  1  R(i-J)  +  "7  a!  +  °n 

N1  q  1  q  N2  q  1  =  1  j  =  l  q  q 


(4-40) 

where  R(n)  =  R  (n)  =  R  (n)  =  R  (n)  +  o^5(n).  Since 

o  i  y  n 

2  4 

cq  >:>  cq  »  (4_4°)  is  approximated  by 

N1  N1 

°e  *  °qINl  *  S7  +  1  *  ^  <Vn  *  ) 

Furthermore,  rr-  <<  N-,  and  since  4  N,a2  <  1  for  reasonable 

1  N2  1  n 

choices  of  N]  and  it  follows  that 


°e  S  aqCNl  +  1  +  -T  1  E  R(i-j)]  .  (4-42) 

q  N:  1=1  j=l  9 


Since  Rg(n)  <_  R  (0)  it  is  easily  shown  that 


N,  N 

2  J  J 

2  L  l 

N  2  i =1  j  =  l 


t(i-j)  <  2  4  (u?  +  o?) 


N,  1=1  j=l  9  9  r 


(4-43) 


where  ug  =  E{g}.  It  is  usually  the  case  that  N2  >  N]  and 

2  2 

since  Ug  <  1  and  j  ^  <  1  it  follows  that  the  error  variance 
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Note  that  if  N ^  =  N  and  Ng  =  1 ,  then  from  (4-49) 

°?o  2  (N  +  2>  °q  (4-51) 

so  the  scaling  of  partial  sums  has  reduced  the  round-off 
error  by  slightly  more  than  a  factor  of  8  in  this  case. 

This  translates  to  a  gain  of  approximately  1.5  bits  of 
effective  precision. 

Floating  Point  Processing 

Floating-point  processing  eliminates  the  need  for 
data  scaling  since  register  overflow  conditions  can  be 
detected  and  handled  by  the  exponent  of  the  floating-point 
number.  For  this  reason,  we  need  only  consider  round-off 
errors.  To  simplify  the  analysis,  each  of  the  two  terms  in 
the  covariance  function  is  analyzed  separately.  First  con¬ 
sider  the  sum  of  products  term 

N 

Ci  =  2  rs  .  (4-52) 

1  i=l  1  1 

The  accumulation  of  error  in  the  computation  of  a  sum  of 
products  has  been  analyzed  (Oppenheim  and  Schafer  1975, 
p.439]  in  the  context  of  quantization  errors  due  to  finite 
register  lengths  in  floating  point  digital  filters.  The  ap¬ 
proach  here  is  similar  although  the  results  developed  here 
apply  specifically  to  correlation. 

With  floating-point  computations  the  errors  are 
signal  dependent  and  may  be  written  as 


(4-53) 
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where  we  assume  that  -  is  a  sample  from  a  distribution  uni 
form  on 


2" b  2-b 

~~  ,  )  where,  again,  b  is  the  available  regis¬ 


ter  length.  As  before,  the  round-off  error  variance  is  given 


by  (4-38) 


If  we  let  { S ^ }  be  the  sequence  of  partial  sums: 


,1,  r<s’' 


(4-54) 


then 


s,  =  r,s,(l  *  s,)  , 

i>2  =  Ir2s2 ( ^  +  ^  +  ^1 J  ^  +  (4-55) 

=  [r2s2  ( 1  +  ^  +  rlsl^  +  +  n2^ 

S3  =  -  r3s3  ( 1  +  £3)  +  +  n3^ 

=  [r3s3(l  +  S3)  +  r^O  +  +  n2^ 

+  r^s-j  (1  +  )(1  +  n2)  1  (1  +  n3) 

where  In)  is  the  sequence  of  addition  round-off  errors  which 
are  also  assumed  to  be  zero-mean  Gaussian  with  variance  given 
by  (4-38). 

Continuing  in  the  above  manner,  it  can  be  shown  that 
N  N 

SN=  Z  s  .  r  .  ( 1  +  5  . )  *  ( 1  +  n • )  ;  n 1  =  0  (4-56) 

N  j=l  J  J  J  i=j  1  1 

From  the  assumptions  on  {5}  and  { n >  we  can  write 


N 

l  A  .  r  .  s  . 


where 


j  =  l 


N 


J  J  J 


(4-57) 


A  •  =  it  ( 1  +  S  i  ) 
J  i  =  j 


(4-58) 


\ 
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and 

we  have  dismissed 

with 

the 

distinction  between 

{ }  and 

{ n } . 

The  cumulative 

round- 

■  off 

error , 

e N ,  is  then 

given  by 

II 

<D 

SN  ' 

-  ci 

= 

N 

z 

i  =1 

flir(5i 

N 

E  r.s. 
1*1  1  1 

(4-59) 

= 

N 

i  - 1 

(A,  - 

1)  Vi 

• 

Taking  expectations  in  (4-59)  w'th  respect  to  round-off 
noise,  we  have 

N 

E(eN)  *  Z  r.s.  E  (A.  -  1)  .  (4-60) 

N  i  =  1  11  1 

From  (4-58),  it  is  seen  that 

N 

E { A  }  =  it  (1  +  E{r})  =  1  (4-61) 

J  i=j 

so  that 

E (eN  >  =  0  .  (4-62) 

The  correlation  output  variance  is  thus  given  by 


Ge  = 
eN  lN 

N  N 

«  E(  E  r.s. (A  -  1)  E  r.s. (A.  -  1))  (4-63) 

i =1  11  1  j=l  J  J  J 

N  N 

=  E  E  r.r.s.s  .  E  { ( A  .  -  1  )(A.  -  1  )} 
i =1  j=l  1  J  1  J  1  1 

Now 

E  ( ( A  i  -  1)(A.  -  1))  =  E(A.Aj )  -  1 

I  and 
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N  N 

E(A . A  . )  =  E  (  -  (1  +  c  )  *  (1  +  £.  )) 

1  J  =i  k= j  K 


=  £  (0  +  sMr(i  +  •; 


N '  11  '  ’N  - 1 


2-.-(i  +  cL)2d  +  C,  _,) 


(1  +  «„)) 


L-l 
(4-64) 


where 

L  =  max( i  ,  j ) 
M  =  m  i  n  ( i  ,  j  ) 

Therefore 


-  N-L+l 

E(AiAj )  =  [E(l  +  CT) 

,  N-max ( i , j )+l 

■  n  * 

It  can  easily  be  shown  that  if  e  <<  1 ,  then 
(l+e)^*Me+l 

Combining  (4-63)  to  (4-67),  we  have 


o  r\  li  li 

■  a.  I  I  r . r  .  s • s  .  ( N  -  max(i,j)  +  1) 

eN  q  i  =1  j  =  l  1  3  1  J 


(4-65) 


(4-66) 


(4-67) 


(4-68) 


The  value  of  ou  clearly  depends  upon  the  input  data 
eN 

sequences.  Reintroducing  the  shift  parameter  k  and  taking 
expectations  over  the  input  processes  yields 


a2  (k)  =  a2  Z  Z  E(r.-r  .s ...  s .  .  )(N  -  max(i.j)  +  1)  . 

eN  q  i=l  j=l  1  J  1  K  J  K 

(4-69) 

As  before,  the  fourth  order  moment  can  be  simplified  for 
zero-mean  Gaussian  processes  for  which  (4-21)  applies.  For 
non-zero-mean  processes,  consider 
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E((rr  +  u)(r:  *  u)(s'  +  a)(s'  *  •„) )  -  Elr/jS.s,,) 

where  we  assume  the  processes  have  identical  means  and  the 
primed  quantities  are  zero-mean.  Since  all  odd  order  mo¬ 
ments  of  zero-mean  Gaussian  processes  are  zero  [Whalen  1971, 
p  .  97  ]  it  foil ows  that 


E(rirjVn)  =  E(rirjsmsn)  +  y  Cr(i'j)  +  Cs  (m‘n)  +  Crs(i‘m) 


+  cr$(i-n)  +  Crs( j-m)  +  Cr$( j-n) 


(4-70) 


Combining  (4-21),  (4-69),  and  (4-70),  we  have 

cl  (k)  =  a2  l  Z  (  C 2 ( i - j )  +  cl  (k)  +  C  (i+k-j)C  (j+k-i) 
eN  q  i=l  j=1  rs  rs  rs 


+  u  (2C(i-j)  +  2Crs(i-j)  +  Crs(j+k-i)  +  Cr$(i+k-j)) 
+  v4  ]  max(i , j) 


(4-71) 


where  C(n)  =  Cr(n)  =  C$(n)  and  max(i,j)  replaces  N  -  max(i-j) 

+  1  due  to  the  symmetry  of  the  autocovariance  functions  and 
the  fact  that  all  terms  depend  only  on  the  difference  i  -  j 
and  not  on  these  individual  values. 

To  complete  the  analysis  of  round-off  errors  in  float¬ 


ing-point  correlators  we  need  to  consider  the  term 


C2  ”  N  E  ri  Z  si 


(4-72) 


If  the  images  are  quantized  to  N  =  2  levels,  then  errors  in 
each  of  the  summations  in  (4-72)  occur  only  as  a  result  of 
overflow.  Thus  to  develop  an  expression  for  the  cumulative 
error  in  (4-72)  we  should  consider  the  probability  that  such 
an  overflow  will  occur  with  each  addition,  or  equivalently, 


the  expected  total  number  of  overflows  that  occurs  in  the 


V 


t 

I 
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computation  of  (4-72).  Even  if  the  distributions  of  r  and  s 
were  accurately  known,  however,  the  computation  of  overflow 
probability  would  be  cumbersome  since  the  likelihood  of  an 
overflow  at  a  given  addition  operation  depends  on  whether  or 
not  an  overflow  occurred  during  previous  additions.  To  avoid 
these  complications,  we  assume  that  round-off  errors  occur 
with  each  addition  and  realize  that  the  result  will  over¬ 
estimate  the  error  variance.  We  thus  consider 


,  N  N 

c2(k)  =  i  z  r.d+^J.r  si+k(1+r,.)  (l+ei)  (l+e2)  (4-73) 


i=l 


i=l 


where  £,  =  =  0  and  e-j  and  e,,  result  from  the  computation 

of  the  product  of  sums  and  division  by  N  respectively.  The 
accumulated  error  in  (4-73)  is  given  by 

N  N 

(4-74) 


1 


S<k>  ■  C2  -  N  ri  T, 


N  N 


N 


N  O-e,  )0+e2)  ^  r.  ^  ^  s(tk  . 


N 

Vi 


N  N 


that 


+  rc:i si+k^  +  h  (erveie2> n  n*k 

(4-75) 

The  statistical  independence  of  error  terms  implies 


\  (k)  *  E(e*(k))  =  X  (1+oJ;)  E[(  z  E  r.s.+.n ,Y 
8s  s  r  q  i=l  j=l  1  3  K  3 


N  N 


N  N 


*  r15j+kM  +  1 


N 


i=l  j=l 

1  2  N  N 

+  -j  Ete^eg+e^g)^  E  (  r  r 


i =1  j=l 


ri  sj*k> 


(4-76) 


Eliminating  all  terms  involving  powers  of  3  greater 
than  2(since  c  4  <<  c  2 )  we  have 


,  ,  ,  N  N  N 

"e  (k)  =  Tcq  :  z  Z 
es  N  q  i  j  m 


Z  E(rirjVk!  +  z  t  ;  z  E(rirjVkW  -(4-77) 

°  i  j  m  n 


N  N  N  N 


Application  of  (4-70)  results  in 


:  00  =  4  ~n  ~  "  (C(i-j)C(O)  +  2C(m+k-j )C(m+k-i 

es  N  q  i  j  m 

+  u2(c(i-j)  +  C(0)  +  rCrs(m+k-i))  +  v2) 


i  j  m  n 


(C(i-j)C(n-m)  +  Crs (m+k-i )Crs(n+k-j) 


+  C  (n+k-i )C  (m+k-j  +  u2(c(i-j)+C(m-n)  +  C  (m+k-i) 


+  Cr$(n+l-i)  +  Crs (m+k-j )  +  Cr$(n+k-j))  +  y4) 


(4-78) 
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Summary  of  Correlation  Noise  Quantities 


At  this  point,  it  is  worthwhile  to  collect  the  various 
error  terms  for  purposes  of  comparison.  The  expression  for 
sensor  noise,  self-noise,  and  machine  noise  are  listed  below. 
Sensor  Noise:  From  (4-9), 


a2  (k)  =  N  a2  ( 2 c2  +  a2) 
e„  n  '  g  n' 

c 


(4-79) 


Sel f-Noi se :  From  (4-23),  removing  the  sensor  noise, 


~  N  N  ~ 

of (k)  -  E  E  (CJi-j)  +  c  (j+k-i)C  (i+k-j)] 
c  1*1  j»l  9  9  9 


(4-80) 


A/0  Quantization  Noise:  From  (4-37), 


2  / 1  >  x  1  2  / p  2  .  2, 

Gca(k)  ‘  N  cq  {2ag  +  °q 

M 


(4-81  ) 


Fixed-Point  Scaling  and  Round-off  Noise:  From  (4-49), 


2  ,  1  2  ,N(  *  "l  +  < 

0  sro  k  N  aq  ^  N]  ' 


(4-82) 


Floating-Point  Round-off  Noise:  From  (4-71)  and 


(4-78)  , 


o2  (k) 
ro '  ' 


■j?  M  *  °l  (k) 


JIN  ,  - 

5  -q  E  z  [C(i-i)  +  C‘s(k)  +  Crs(i+k-j)Crs(j+k-i) 
^  3 


+  /(2C(i-j)  +  2Cr$(i-j)  +  Crs ( j+k-i ) 


+  Cr$( i+k-j))  +  u  ]max(i ,j) 


(4-83) 

( contd  ) 
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L  O 

+  Z  Z  I  C(i-j)C(0)  +  2C(m+k- j )C(m+k-i ) 

N  1  j  m 

+  u 2 ( c ( i - j )  +  C(0)  +  rCrs(m+k-i))  +  u4 

+  r  :  :  :  C(i-j)C(m-n)  +  C  (m+k-i)C  (n+k-j) 
i  j  m  n 

+  Crs^n+k'i'(Wm+k"J') 

+  u2(C(i-j)  +  C(m-n)  +  Crs(m+k-i)  +  Cr$(n+k-i) 

+  Crs(m+k-j)  +  Crs(n+k-j)  +  u4 

(4-83) 

with  the  assumptions  that 

C ( n )  =  Cg(n)  +  c2  t ( n ) 

Crs(n)  =  Cg(n) 

(4-84) 

and  u=E(r)=E(s)=E(g) 


2  2 

The  signal  dependence  of  a  (k)  andcrQ(k)  and  the  com¬ 
plexity  of  (4-33)  make  it  difficult  to  ascertain  the  relative 
magnitudes  of  the  noise  terms  for  arbitrary  source  auto¬ 
covariance  functions.  We  must  also  note  that  the  fixed  point 


errors  resulted  from  the  computation  of  c ^ / N  rather  than 


c  ^  so  any  comparison  must  account  for  this  factor  by  multi- 
2  2  2 

plying  :c^(k)  and  ^srCj(k)  by  N  .  Equivalently  we  could  com¬ 
pute  the  correlator  output  SNR  given  by 


A(k) 


(4-85) 
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2 

where  _*  (k)  represents  any  of  the  noise  terms.  There  is 

1 

2 

some  question  as  to  whether  /.  or  should  be  called  the  out- 

1 

1 

put  SNR.  The  argument  in  favor  of  is  that  the  correlator 

output  is  actually  a  "power  signal".  We  have  selected  A, 
however,  since  it  represents  the  output  signal-to-noise 
power  ratio,  regardless  of  the  physical  units. 

For  our  example,  consider 


cg(n)  ■ 


exp 


I  1 


i  <?) 


(4-86) 


where  i  represents  the  correlation  width.  Since  the  noise 
and  signal  processes  have  been  assumed  to  be  Gaussian  on 
the  interval  (0,1)  (see  (4-31)),  we  will  let  =  .05, 

2-4  2 

_  n  -  .5,  and  ;  =  5  x  10  .  This  value  of  a  implies  that 

9  n  g 

about  3%  of  the  total  area  of  the  Gaussian  distribution  lies 
outside  the  unit  interval.  The  input  signal-to-noise  ratio 
is  20aS.  We  also  select  N  =  64  and  N-j  =  8  since  they  are 
typical  values  employed  by  the  AS-11B-X  compiler.  The 
correct  peak  output  SNR's  (at  k  =  0)  associated  with  each 
noise  source  are  plotted  in  Figure  4-2  for  the  selected 
parameters.  For  this  example,  it  appears  that  self-noise  is 
the  dominating  error  source  regardless  of  the  correlation 
width.  This  result  is  misleading,  however,  because  the 
nature  of  self-noise  differs  significantly  from  the  other 
noise  sources.  The  self-noise  curve  in  Figure  4-2  represents 
the  expected  amount  of  fluctuation  in  the  magnitude  of  the 
peak  of  c ro s s - co va r i a  nee  function  due  to  the  random  nature 
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\ 


100 


of  the  underlying  image  formation  process  within  a  region  of 
imagery  that  is  assumed  to  be  characterized,  in  the  statis¬ 
tical  sense,  by  the  autocovariance  function  C  .  The  remain¬ 
ing  SNR  curves,  on  the  other  hand,  represent  the  random  flue 
tuation  in  peak  correlation  magnitude  at  a  given  location 
due  to  the  random  nature  of  the  noise  source.  A  second  fund 
amenta!  difference  between  self-noise  and  the  other  noise 
sources  is  that  the  self-noise  cross-covariance  between  taps 
increases  with  A.  Figure  4-3  shows  the  relationship  between 
the  normalized  autocovariance  function  and  the  normalized 
covariances  between  the  center  tap  and  all  other  taps  for 
N  =  16  and  A  =  1.,  5.,  10.,  and  20.,  assuming  the  autocovar¬ 
iance  function  of  (4-86).  For  this  particular  example,  the 
tap  covariances  are  slightly  greater  than  the  input  signal 
covariances  for  identical  function  sample  separations,  i.e., 
E(CqC£)  >  E(gkgk+£)  U>  0  .  (4-87) 


These  properties  of  self-noise  make  it  difficult  to 
determine  the  contribution  of  self-noise  to  correlation 
error  production.  Since  our  goal  is  the  development  of 
usable  image  quality  measures,  an  experimental  approach 
based  on  the  probability  of  false  acquisitions  is  developed 
in  the  following  section.  Two  image  quality  features,  one 
of  which  ignores  the  contribution  of  self-noise,  are  devel¬ 
oped  using  (3-39).  The  performance  of  the  measures  in  a  de¬ 
tection  scenario  will  be  discussed  in  Chapter  6. 


\ 
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Figure  4-3. 


Center/Side 


Tap  Covariance 


N  =  16 

1  =  data  autocovariance  function 

2  =  center/side  tap  covariance 
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False  Acquisition  Measures 
Equation  {3-39}  defines  an  image  quality  measure 

a  .  ^  m-88) 

where  -  (  k )  is  tne  correlator  output  standard  deviation  at 
tap  k  due  to  noise  injection  of  the  various  types  aescribed. 
For  the  present,  we  will  ignore  quantization  noise  and  machine 
noise  (simulations  nave  been  performed  using  floating  point 
aritnmetic  with  34  bit  precision  which,  provides  an  SNR  of 
over  100  d3)  and  consider  two  expressions  for  Q  based  on 
sensor  noise  onlv  for  wnich 


;c(k)  =  ’N  J n  {2z\  +  -n 


1 

7 


(4-89) 


'and  for  sensor  noise  plus  self-noise  for  which 

Vki  *  !»  =n!2=9  *  Jn]  *  ::  C9(i'j! 


+  c  (  i  +  k  -  j  )  C  g  (  j  +  k  -  i  )  ]' 


(4-90) 


whicn  for  larce  k  becomes 


(k)  = 


•  f  4>  *  =:c9(i-J))2 


ror  white  noise  input,  (4-91)  becomes 

:  ( k )  =  v  N  (  2  +  z2 )  =  kN  a2  =  -fie2 

Cv  '  g  ‘  5  p 


.  (4-91) 


(4-92) 


wnere  the  model  of  (3-7)  with  a  =  1  is  assumed.  Since  the 
contribution  of  self-noise  to  error  production  is  unclear, 
and  to  avoid  the  computation  of  the  autocovariance  function 
(which  may  not  be  a  good  statistical  representa ti on  of  cor¬ 
relation  behavior  when  estimated  over  small  data  sets),  we 
introduce  the  factor  x  and  let 


where 


4 

X  N  a  + 
9 


,22. 
2a  + 
n  g 


(4-93) 


0  <  X  <  1 

since 

A  N  N  ,  ,  a 

0  <  w-l  <  1  l  C„(i-j)  <  N  .  (4-94) 

~  9  “  i=1  j=i  9  9 

If  X  =  0,  (4-93)  reduces  to  (4-89)  and  if  X  =  (4-93)  re¬ 

duces  to  (4-92).  While  there  are  a  variety  of  ways  to  make 
X  adaptive  (such  as  computing  (4-90)  directly)  we  consider 
the  expression 


*  ■  1  ♦  Vf  -  >> 


(4-95) 


where 


xo  6< 


2  2 

ar  "  °n  . r  2  2 

- r~  lf  °r  "  an 


otherwise 


(4-96) 


This  choice  of  x  is  motivated  by  the  reasonable  assumption 
that  low  variance  imagery  typically  displays  a  broad 
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autocorrelation  function  whereas  high  variance  imagery  tends 
to  generate  narrow  autocorrelation  functions.  Substituting 
(4-93)  into  (4-88)  yields 


_ /N  SNR _ 

I  A N ( S N R  )  2  +  2SNR  +  1  ]  1  /  2 


(4-97) 


2  2 

where  SMR  =  c  / c  is  the  input  sicnal-to-noise  ratio.  Since 
g  n 

2 

:  must  be  estimated  from  r  (or  s)  we  compute 


2  2 

'  -  c 

r  n 


2  2 
°r  >  °n 


SNR  = 


(4-98) 


0  ;  otherwise 

Figure  4-4  displays  the  single  tap  false  acquisition 
probability,  PpA,  as  a  function  of  Q.  The  behavior  of  Q  as 
a  function  of  SNR  and  a  is  shown  in  Figure  4-5  for  N  =  21 
and  64.  Note  that  the  larger  fixed  values  of  A  (say  \  >  .5) 
do  not  result  in  reasonable  \.ur  observed)  values  of  PpA>  For 
example,  for  a  =  .5,  the  single  tap  false  acquisition  prob¬ 


ability  cannot  be  improved  beyond  P, 


1  regardless  of  the 


input  SNR  and  window  size.  While  P  p  A  is  only  an  indication 
of  the  true  total  false  acquisition  probability  Py  (over  all 
taps),  it  is  necessary  that  PT  >  PFA- 

Regardless  of  how  Q  is  computed,  there  is  a  one-to- 
one  relationship  between  SNR  and  PpA<  Thus  it  is  reasonable 
to  consider  the  alternative  quality  measure  given  by 


SNR 


(4-99) 
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2 

which  is  also  shown  in  Figure  4.5.  If  j  is  a  known  constant 

n 

then  Q i  varies  directly  with  the  local  image  variance. 

The  use  of  the  quality  measures  Q  and  in  a  threshold  de¬ 
tection  algorithm  will  be  discussed  in  Chapter  6.  It  is 
clear,  however,  that  the  choice  of  Q  and  A  alters  the  pdf  of 
the  quality  measure  which,  in  turn,  affects  the  selection  of 
appropriate  threshold  levels. 


Estimated  Mean  Square  Error 


A  closed  form  expression  for  the  estimated  mean  square 
error  defined  by  (4-3)  can  be  derived  by  extending  the  result 
in  (3-36)  if  we  assume  that  the  correlator  tap  outputs  are 
statistically  independent.  If  such  is  the  case,  then 


L 

7T 

n  =  -L 
r\tt 


Pr  (y)dy  p  ( x ) dx 
un  '  L  ' 


'l 


where  p  (x)  is  the  correlator  output  pdf  at  tap  k. 
c  k 

the  assumption  of  Gaussian  pdf's,  (4-100)  becomes 


(4-100) 


Under 


P  ,  =  f  n 

£  J  n=-L 
n  tl 


u (x , n ,£) 


Gaus(y)dy  Gaus(x)dx  (4-101) 


where 


3_U)x  +  N(C  U)  -  C  (n) ) 
u  (  x  ,  n  ,  £ )  =  -  rs  rs 


ccW 


(4-102) 


Computation  of  (4-101)  generally  leads  to  overestimates  of 
MSE  because  the  tap  covariances  are  neglected.  This  can  be 
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counteracted  to  some  degree  b}  imposing  the  constraint  that 
[Ryan  and  Hunt  1 980] 

L 

Z  P.  =  1  .  (4-103) 

£  =  -L  ^ 

The  computational  complexity  of  (4-101),  however,  makes  it 
cumbersome  to  use  as  an  image  quality  feature. 

Input  Data  Non-Stationarity 

Virtually  all  of  the  literature  on  correlation 
behavior  ignores  the  effects  of  non-stationarity  of  input 
data.  Each  of  the  expressions  derived  in  the  preceding  sec¬ 
tions  require  the  assumption  of  stationarity  to  express  the 
expected  behavior  in  terms  of  ensemble  statistical  averages 
(means,  variance,  autocorrelations,  etc.).  To  make  use  of 
these  results,  however,  requires  estimating  the  statistical 
averages  from  finite  data  records  which,  in  turn,  requires 
that  the  input  data  be  both  stationary  ana  ergodic.  Images, 
however,  are  notoriously  non-stat ionary.  The  local  statis¬ 
tics  within  a  windowed  image  can  change  drastically  as  the 
window  moves  from  one  location  to  another  particularly  if 
the  window  passes  an  edge  (a  local  discontinuity  in  image 
luminance  or  amplitude  level)  between  reasonably  smooth 
regions.  Many  image  processing  algorithms  (for  restoration, 
enhancement,  etc.)  are  improved  by  adapting  algorithm  para¬ 
meters  to  local  image  statistics  to  account  for  the  non- 
stationarity  of  the  image.  Improvements  in  correlation 
accuracy  can  be  achieved  in  a  similar  manner  as  the  following 
discussion  implies. 


The  assumptions  leading  to  covariance  as  the  ML  proc¬ 
essor  include  the  assumption  that  the  variance  (or  energy) 
of  the  search  area  sub  image  does  not  change  appreciably  over 
the  search  area.  If  this  assumption  is  violated  due  to  the 
presence  of  an  edge  within  the  search  area,  the  resulting 
behavior  of  the  covariance  function  is,  in  many  cases,  pre¬ 
dictable  as  the  following  example  implies. 

Consider  the  sequence  of  picture  element  values  shown 
in  Figure  4-6.  We  select  from  this  sequence  a  match  window 
of  length  5  centered  on  position  2 .  This  match  window  is 
thus  the  sequence  {4, 6, 4, 5,8}.  This  window  is  then  corre¬ 
lated  with  the  entire  sequence  using  the  covariance  function 
as  a  measure  of  similarity.  The  resulting  correlation  func¬ 
tion  is  shown  in  Figure  4 - 7 a .  The  location  of  the  correct 
correlation  peak  is  denoted  by  location  0.  The  peak  ac¬ 
tually  occurred  at  location  4  so  there  is  a  registration 
error  of  4  pixels.  Figures  4  -  7  b  to  4  -  7  j  display  similar 
results  for  all  other  match  window  positions  selected  from 
Figure  4-6.  All  match  window  positions  chosen  have  maximum 
cross-covariance  corresponding  to  center  positions  6,  7,  or 
8  in  Figure  4-6.  These  points  form  the  transition  region 
(edge)  between  two  regions  in  which  the  statistics  change 
much  more  slowly.  The  only  correct  correlation  results  in 
this  experiment  occurred  for  match  window  center  positions 
7  and  8.  Since  covariance  removes  the  means  from  the  se¬ 
quences,  the  covariance  processor  can  be  thought  of  as 
imposing  stationarity  of  the  mean  upon  the  process.  However, 


Figure  4 -  7 .  Covariance 


Figure  4-7  (continued).  Covariance  Functions  for  the  Sequence  in  Figure 
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no  correction  is  made  for  non-s ta tionar i ty  of  the  variance. 
Normalized  covariance,  however,  imposes  stationarity  of  the 
variance  upon  the  process  by  normalizing  the  signal  energy 
at  each  position.  As  seen  in  Figure  (2-7),  the  error  be¬ 
havior  appearing  in  Figure  (2-5)  does  not  occur.  Normalized 
covariance  is  a  special  case  of  generalized  1 ea s t- squa res 
given  in  (2-20)  so  we  expect  least  squares  to  behave  in  a 
manner  similar  to  normalized  covariance  in  the  vicinity  of 
an  edge.  Figures  2-10  and  2-11  indicate  that  this  is,  in¬ 
deed  ,  the  case . 

A  close  comparison  of  Figures  2-9  and  2 - 4 b  (by  using 
an  overlay)  shows  that  there  are  regions  of  considerable 
error  located  along  the  edges  of  Figure  2  - 4  b .  It  is  also 
clear  that  positive  errors  generally  lie  on  one  side  of  an 
edge,  while  negative  errors  lie  on  the  opposite  side.  This 
behavior  is  also  noted  in  Figure  4-6.  It  is  thus  reasonable 
to  conclude  that  at  least  some  of  the  errors  occur ing  in 
Figure  2-9  as  a  result  of  covariance  processing  are  due  to 
the  presence  of  edges  which  violate  the  assumption  of  image 
stationari ty. 

We  present  here  two  approaches  to  the  detection  of 
such  regions.  For  stereo-pair  images  with  negligible  relief 
distortion,  the  value  of  the  covariance  function  at  the  cor¬ 
rect  peak  location  is  an  estimate  of  the  energy  within  the 
image  region  under  observation.  As  before,  we  assume  the 
distribution  of  image  energy  is  Gaussian,  i.e., 


(4-1 04) 


The  presence 
resul ts  in  a 
local  energy 
distribution 


(x)  =  -  expl^  ->C(Q)?'l 

v'  2  -  o  c  ( 0  )  2^(0) 

of  an  edge- produced  region  of  no n - s ta t i o na r i ty 
large  peak  covariance  value  due  to  the  high 
(or  variance)  across  the  edge.  Thus  given  the 
of  (4-104),  we  can  select  a  threshold  t  such 


that 


Prob(CQ  >  t]  =  |  pe(x)dx  (4-105) 

t 

is  some  desired  value.  Such  a  detection  scheme  could  be 
implemented  either  as  a  preprocessor  or  as  a  reliability 
measure  used  during  the  stereo  compilation  procedure  itself. 
Detection  of  these  regions  would  enable  an  adaptive  processor 
to  convert  to  normalized  covariance  or  least-squares  corre¬ 
lation  in  the  region  of  expected  difficulty.  The  processor 
returns  to  covariance  processing  when  the  numerator  of  the 
normalized  covariance  function  or  one-half  of  the  first  term 
in  least-squares  correlation  falls  below  the  selected  thresh¬ 
old. 

It  is  also  interesting  to  consider  the  conditions 
under  which  covariance  produces  errors  and  1 e as t- sq ua r es 
(and  presumably  normalized  covariance)  produces  correct  peak 
locations.  Suppose  the  correct  hypothesis  (using  the  nota¬ 
tion  of  Chapter  2)  is  H  .  but  that  covariance  selects  H  ^  . 

This  condition  implies  that 
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(covariance):  r's^^-r's^1^->0  (4-106) 

(least  squares):  2r's^  -  s^1^  s^1^  <  2  r  s  '  s  ( j  )  -  s  '  ^  s(j) 

(4-107) 


where  we  assume  the  means  have  been  previously  removed. 
Equations  (4-106)  and  (4-107)  can  be  combined  to  yield 

sij)  s  v J )  _  s ( A  s  ^  >  2(r  s  ^  -  r  s  ^  ^ )  >  0  .(4-108) 
This  condition  simply  states  that  the  covariance  processor 
can  select  H  .  only  if  the  energy  of  sub  image  s  ^  is  gr  iter 
than  the  energy  of  subimage  s  ^  ^  .  It  is  at  least  intuitively 
reasonable  that  the  quantities 


_  L  A j):(j)  . ( i ) '  5  ( i ) 

A  3  "  S  S  ‘  S  S 


(4-109) 


and 


o  .  ,  ^  2r'(s(j)  -  s(l  h  (4-110) 

1  J 

are  correlated  so  that  as  e.j  increases,  the  probability 
that  >  0  also  increases.  If  the  distribution  of  £.. 

*  V  ’  vJ 

conditioned  on  c  •  ■  ,  p ( 5  -  - ; e -  . )  were  known,  it  would  be 
i  J  i  J  i  J 

possible  to  establish  thresholds  on  c  .  ^  which  would  allow 
the  detection  of  image  regions  for  which  Prob  U  .  .  >  0]  ex- 

1  J 

ceeds  a  maximum  acceptable  error  rate.  Such  a  distribution 
could  be  determined  experimentally  with  sufficient  computing 
power  and  a  set  of  " representa ti ve"  images.  A  processor 
which  might  be  employed  in  such  a  detection  scheme  is  shown 
in  Figure  4-8. 
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Since  the  computation  of  p(::e)  may  be  computationally 
prohibitive,  a  more  realistic  approach  can  be  developed  by 
noting  that  c  is  related  to  the  gradient  of  the  image  energy. 

A  simplified  pre-processor  based  on  this  realization  is  shown 
in  Figure  4-9.  In  Chapter  6,  we  present  experimental  results 
to  indicate  the  feasibility  of  such  a  procedure.  This  dis¬ 
advantage  of  this  approach  is  that  the  resulting  error  map 
indicates  only  the  locations  of  points  corresponding  to  high 
error  probabilities  and  information  concerning  the  error 
magnitude  is  lost.  Such  a  binary  error  map  could  be  used  to 
switch  the  compilation  processor  from  covariance  to  a  more 
reliable  technique,  and  this  may  be  all  that  is  required. 

The  Extension  to  Two  Dimensions 

The  various  correlator  output  noise  variances  given 
in  (4-79)  to  (4-83)  were  derived  under  the  assumption  of  one¬ 
dimensional  source  sequences  and  a  one-dimensional  search. 

Of  these  quantities,  only  self-noise,  (4-80),  and  floating¬ 
point  round-off  noise,  (4-83),  are  dependent  on  the  signal 
statistics  and  thus,  on  the  shape  of  the  match  window.  Since 
both  of  these  expressions  result  from  the  expansion  of  the 
fourth  order  moment  given  in  (4-21),  the  extension  to  two- 
dimensional  simply  requires  the  corresponding  expression  for 
the  two-dimensional  case.  Beginning  with  the  2-D  expression 
for  the  covariance  function  given  in  (4-5)  and  ignoring  the 
sensor  noise,  we  have 


c(k,£)  = 
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i=l  j=l 
,  M  N 


9i » j ' i  +  k , j  +  £ 
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Assuming,  as  before,  that  the  error  in  estimating  the  mean 
is  negligible,  we  let  g  be  zero-mean.  Thus 

M  N  M  N 

£'c{k,£)c(*,n))  •  iE)  £  ^  b;)  Si.jVk.j+A.bW*-!  • 

(4-112) 

But  the  two-dimensional  counterpart  of  (4-21)  results  in 
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+  Cg_(a-i,b-j)Cg(aJ-m-i-k,b  +  n-j-£) 


+  C 


g(a+m-i,b+n-j)Cg(i+k-a,j+£-b) 


The  individual  tap  variances  result  from  (4-113)  and  (4-111) 
when  k  =  m  and  l  -  n  for  which 


a^(k,£)  =  E(c2(k,£))  -  E2(c(k,£)) 

M  N  M  N  - 

3  Z  z  Z  Z  C^(a-i,b-j)  (4-114) 
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CHAPTER  5 


RELIEF  DISTORTION 

In  all  of  the  preceding  discussion,  we  have  made  the 
assumption  that  topographic  relief  is  negligible  so  that  the 
two  image  functions  differ  only  by  additive  noise  terms; 
that  is,  the  underlying  image  functions  are  identical.  The 
presence  of  topographic  relief,  however,  introduces  distor¬ 
tion  between  the  two  image  functions.  The  primary  effect 
of  this  distortion  is  a  reduction  in  the  peak  value  of  the 
cross-correlation  function  which,  in  turn,  increases  the 
probability  of  both  local  registration  and  false  acquisition 
errors.  In  this  chapter,  a  model  of  relief  distortion  is 
developed  and  used  to  determine  the  effects  of  relief  dis¬ 
tortion  on  the  covariance  function  and  on  the  image  quality 
measures  previously  discussed. 

Relief  Distortion  Model 

Consider  the  image  formation  geometry  shown  in  Figure 
1-2.  The  image  generation  equations  were  given  in  Equations 
(1-2)  and  (1-3)  and  repeated  here: 

x,(H  -  e ( X  ) ) 

g  ]  (  x  i  )  -  I(  — - 2 - )  (  5-1  a  ) 

x o ( H  -  e( X) ) 

g 2 ( x2 )  =  1(5  -  — - j - )  .  (5-lb) 
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(5-2) 


where  X  g  is  as  shown  in  Figure  1-2,  then  x-j  and  x  2  are  con¬ 
jugate  points  and  g^(x^)  =  g2(x2).  ^  there  is  n0  terra'’n 

relief,  then  9-](xi  +  A  x )  =  g2(x2  +  Ax).  order  to  deter¬ 

mine  the  effect  of  distortion  on  the  image  functions,  con¬ 
sider 

( x i  +  Ax  )  ( H  -  e(X^) 


9  (  x  -j  +  Ax 


=  I 


=  I ( X 1 )  ( 5 - 3a  ) 


and 


9  2 ( x  g  +  ax2)  =  1 


(x?  +  Ax?) (H  -  e(X?) 

3  +  - 2_j  =  !  (x2)  (5-3b) 


where  +  Ax-j  and  x^  +  ax2  are  the  images  of  terrain  points 
X i  and  X^  respectively.  We  wish  to  determine  the  value  of 


ix2  so  that  X i  =  X 2  =  X 


After  some  algebraic  manipulation 
involving  (1-1),  (5-2),  and  (5-3),  it  follows  that 


^x  ,  =  Ax  , 
c  1 


1  -  B 


e  (  X  )  -  e  ( X  0 ) 

XHg  -  Xq(H  -thot 


(5-4) 


where  Hg  =  H  -  e ( XQ )  . 


Letting 
D(X)  * 

we  can  rewrite  (5-4)  as 


e(X)  -  e  (  Xg ) 
X  -  X. 


(5-5) 


Ax2  =  A  X i 
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Finally,  if  we  let 
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and 


f 1  (x)  =  g1 (x1  +  x ) 


f2(x)  =  g  2 ( x  2  +  x 


then  the  model  for  distortion  can  be  written  as 

f -I  ( x  )  =  f  2  (  h  (  x  )  ) 

where 

h  (  x  )  =  x 

and  x  is  rel a  ted  X  by 

y  _  x  ( H  -  e  ( X  ) ) 

A  p 


1  p  D(X) 

B  Hq  +  XgD(X) 


(5-7) 


(5-8) 


(5-9) 


(5-10) 


Equation  (5-8)  expresses  the  effects  of  relief  distor 
tion  in  terms  of  the  functional  composition  [Apostal  1974] 
of  an  image  function  with  a  distortion  function  h(x).  Now 
suppose  that  C(x)  =  m  where  m  is  a  constant  specifying  the 
terrain  slope.  Then 


h  (  x  )  =  x  1 
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(5-11) 


It  is  usually  the  case  that  Hq  >>  mX  so  that 

h  (  x  )  =  x 
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(5-12) 


This  result  implies  that  constant  terrain  slope  produces 
spatial  contraction  or  expansion  of  one  image  function  with 
respect  to  the  other.  The  value  of  i  depends  on  the  base- 
height  ratio,  3/ H  ,  and  the  magnitude  of  the  terrain  slope 
m ' ,  with  distortion  increasing  with  increases  in  either  of 
these  parameters. 


The  Effect  of  Distortion 
on  Correlation  Accuracy 


The  effect  of  signal  distortion  on  the  cross-correla¬ 
tion  accuracy  is  most  easily  analyzed  in  the  continuous  do¬ 
main.  Consider  the  cross-covariance  function 


D 

2 

r(-)  =  :  f  (  x  +  -; )  g  (  x  )  d  x  (5-13) 
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where  0  is  the  length  of  the  match  window  and  f  and  g  are 
sample  functions  from  stationary,  zero-mean  Gaussian  processes. 
If  g  ( ;< )  =  K f  (  x  -  x  r )  , 

then 


D 

2 

E(r(-;))  =  K  |  Cf  ( •;  +  xQ)dx  =  KDCfU  +  xQ)  (5-14) 
0 

'2 

and  the  expected  peak  location  is  the  translation  between 
the  functions  f  and  g  as  desired.  If  g ( x )  £  Kf ( x  -  x  q  )  for 
fixed  but  arbitrary  K  and  Xq,  then  we  refer  to  g  as  a  dis¬ 
torted  version  of  f,  assuming,  of  course,  that  f  and  g  are 
similar  and  that  true  points  of  conjugacy  exist. 

’.*J e  now  assume  the  distortion  model  of  (5-3)  and  let 

g  ( x  )  =  f ( h ( x ) )  .  (5-15) 

Without  loss  of  generality,  let  K  =  1  and  Xq  =  0  so  that  the 
presence  of  distortion  implies  h(x)  f  x.  Substituting  (5-15) 
into  (5-13)  and  taking  expectations  gives 
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5-16) 


D 

2 

E  (  r  (  •; )  )  =  ■  Cf(  ;  +  x  -  h  (  x  )  )  d x 

D 

'2 


If  u(x)  -  h(x)  -  x  is  a  monotonic  function  of  x,  then  there 
exists  a  function  z  with  z(u)  =  x  such  that  (5-16)  can  be 
written  as 


E ( r ( £  ) )  =  '  ,  z  '  ( u ) [ rect( u  ~  u)Cf ( ;  -  u)du 


5-17) 


where  z'(u)  =  dz/du, 


rec  t ( x  )  =  < 


1  for  |  x  !  < 


0  otherwise 


h(§)  +  h(-f) 


and 


D  +  h(-J)  -  h(J) j 


(5-18) 


(5-19) 

The  derivation  of  (5-17)  is  presented  in  Appendix  B.  The 
monotonic  restriction  on  u ( x )  implies  that  the  distortion 
consists  of  spatial  compression  or  spatial  expansion  but  not 
both.  For  aerial  imagery  taken  over  reasonably  smooth 
terrain,  the  restrictions  are  minor.  If  u(x)  is  not  mono¬ 
tonic,  then  tiie  resulting  ensemble  correlation  function  can 
be  determ inea  directly  from  (5-16).  Equation  (5-17)  is  the 


convolution 
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E(r(-;))  =  ;  z  '  (  u  )  ■  i  rect(^)  *  D  Cf(-;)  (5-20) 

which  shows  that  the  effect  of  relief  distortion  is  to  smooth 
the  correlation  function  and  reduce  its  peak  value.  equation 
(5-20)  also  implies  that  the  peak  of  the  correlation  function 
may  be  biased  (u  t  0)  so  that  the  choice  of  the  true  correla¬ 
tion  peak  results  in  incorrect  matching  of  imagery.  The 
fluctuation  of  u  provides  a  limit  on  the  attainable  accuracy 
even  if  the  imagery  is  noise-free.  It  is  straightforward  to 
show  that  under  the  assumptions  leading  to  (5-12), 

E(u)  =  0  ( 5-21  a ) 

E(iJ2)  =  c2  =  G2  (5-21  b) 

u  a 


where 


Equation  { 5 - 2 1 b )  i m d 1 i e s  that  the  registration  bias  flue- 
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likely  that 

2  •  ,  ,  , 
a  will  also  in- 

ct 

crease. 

Thus  while  a  larg 

er  match  window 

provides  better 

noise  suppression,  it  increases  the  susceptibility  to  regi¬ 
stration  bias  e  r  r  o  r  s  . 

For  arbitrary  terrain,  the  function  z(-)  in  (5-17) 
may  be  impossible  to  obtain  in  closed  form  (it  may  not  be 
unique;.  It  is  obtainable,  however,  for  the  special  but  use¬ 


ful  case  in  which  h(x)  is  given  by  (5-12). 


Substitutina 


E(r;:))  =  7T-5-T;  re:t  (DT (5-25) 

in  which  the  location  of  the  peak  is  certainly  ambiguous, 
but  most  importantly,  the  peak  height  is  reduced.  The  re¬ 
duction  is  most  significant  for  large  values  of  the  product 
D : a  -  1!  and  for  imagery  containing  high  spatial  frequency 
energy  resulting  in  a  narrow  autocorrelation  function.  The 
distortion  characterized  by  h(x)  =  xx  is  analogous  to  the 
doppler  shift  which  occurs  in  radar  systems  when  there  is 
relative  motion  between  the  target  and  signal  sou rce - r ece i ver 
[Skol  ni  k  1  9  52]. 

The  effect  of  the  affine  transformation  h(x)  =  a  x  on 
correlator  performance  has  been  analyzed  by  Mostafavi  and 
Smith  [1973a,  1978b] .  They  also  consider  a  more  general 
af fine  transformation  which  includes  a  possible  rotation  be¬ 
tween  conjugate  regions. 

In  radar  systems  the  effect  of  a  doppler  shift  on 
the  matched  ^ i 1  ter  mismatch  is  usually  considered  to  be  negli¬ 
gible  unless  the  lengthening  or  shortening  of  the  signal  is 


on  the  order  of  the  inverse  of  the  signal  bandwidth.  In 
other  words,  the  shift  (distortion)  can  be  ignored  if 


where  ID  is  the  change  in  signal  extent  and  is  the  signal 
bandwidth.  In  the  present  instance,  ! D  =  (1  -  a )  D .  We  will 
thus  define  the  distortion  measure  as 

d  ^  BgD; 1  -  a!  .  (5-27) 

Continuing  the  analogy  between  the  effects  of  doppler  distor¬ 
tion  and  relief  distortion,  we  require  that 

d  <<  1  .  (5-28) 

In  Chapter  6,  simulations  of  the  distortion  modelled 
by  h(x)  =  :x  show  that  (5-28)  also  applies  in  image  cross¬ 
correlation  applications.  The  use  of  (5-27)  and  (5-28)  as  a 
predetection  feature  depends  on  the  ability  to  estimate  a 
from  the  stereo-pair  without  actually  performing  the  stereo 
compilation  procedure.  Although  such  an  estimation  procedure 
will  be  considered  shortly,  it  is  more  practicable  to  obtain 
a  space-bandwidth  map  of  the  images  ( i  .  e .  ,  a  matrix  contain¬ 
ing  S^D  values  correspond i ng  to  each  image  point),  then  com¬ 
pute  the  distortion  measure  d  during  the  compilation  process 
where  a  (cr  more  generally  h(x))  can  be  obtained  from  succes¬ 
sive  parallax  values.  Values  of  d  exceeding  a  pre-established 
threshold  indicate  regions  of  probable  correlator  performance 
degradation.  Improved  correlation  can  be  obtained  by  warping 
the  appropriate  image  functions  according  to  (5-15)  (effective¬ 
ly  forcing  a  to  be  equal  to  1).  An  algorithm  for  the  on-line 
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spatial  warping  of  image  patc.nes  has  been  described  by  Panton 
[1978].  Such  an  on-line  procedure  either  requires  interpola¬ 
tion  on  previously  stored  data,  or  rescanning  the  image  with 
updated  sampling  rates  based  on  the  expected  terrain  slope 
at  the  next  match  location.  Increased  throughput  rate  could 
thus  be  achieved  by  performing  the  warping  only  when  required 
as  specified  by  the  distortion  measure  d. 

Webber  and  Qelashmit  [1974b]  have  considered  the  fil¬ 
tering  of  imagery  to  desensitize  the  correlation  process  to 
linear  scale-factor  distortion  described  by  h(x)  =  ax .  They 
conclude  that  for  low  SNR  and  Gaussian  autocorrelation  func¬ 
tions  given  by  (4-86)  that  a  should  be  adjusted  by  filtering 
so  that  :  =  D  a  j  /  /6 .  This  desensitization  process,  however, 
apclies  only  to  local  registration  accuracy.  Furthermore, 
as  we  have  previously  stated,  low  SNR  imagery  typically  re¬ 
sults  in  broad  autocorrelation  functions  which  are  already 
insensitive  to  distortion. 

As  an  example  of  the  effect  distortion  can  have  on 
t n e  expected  correlation  function,  consider  the  autocovariance 
function  used  in  previous  examples: 


Cf  ( -;)  =  exp  [-■£(£)  2  ] 


(5-29) 


Using  (3-14),  the  bandwidth  associated  with  this  auto¬ 
covariance  function  is  given  by 
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Evaluating  (5-20)  for  h(x)  =  ix  leads  to 


(5-30) 
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(5-31  ) 


Expressing  this  result  in  terms  of  the  distortion 
leasure  d,  we  have 
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(5-32) 


Figure  5-1  displays  some  representative  correlation  curves 
for  several  values  of  the  parameter  d.  The  effects  of  prime 
importance  are  the  reduction  in  peak  height  which  effects  the 
false  acquisition  rate  and  the  alteration  in  the  curvature 
of  the  function  at  the  origin  which  affects  the  local  regi¬ 
stration  accuracy  (see  3-19).  The  reduction  in  normalized 
peak  magnitude  and  the  reduction  in  normalized  curvature 
magnitude  are  plotted  in  Figure  5-2  as  a  function  of  d.  These 
effects  have  been  observed  experimentally  by  Casasent  and 
Psaltis  [1976]. 


The  Effect  of  Distortion 
on  Error  Prediction 


In  Chapter  3,  we  saw  that  the  local  registration  accu¬ 
racy  depends  on  the  second  moment  of  the  image  power  spectrum 


Normalized  Magnitude 
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in  the  manner  described  by  (3-11).  From  (3-19)  and  (3-11) 
we  see  that 


EU 
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d2Cq(0! 

d^2  ! 
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(5-33) 


Since  the  curvature  of  the  c ros s - co va r i a nc e  function  decreases 
with  increased  distortion,  a  correction  factor,  k ^  ,  can  be 
included  in  (5-33)  to  obtain 
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(5-34) 


where  k-j  depends  on  the  form  of  the  autocorrelation  function 
and  the  distortion  measure  d.  For  the  Gaussian  autocorrela¬ 
tion  function,  k ^  is  obtained  from  Figure  5-2  and 

1,  H^2 

ti  d ) 

k]  =  e^  .  (5-35) 

Equation  (5-35)  describes  the  approximate  increase  in  the 
Cramer-Rao  lower  bound  due  to  the  presence  of  distortion. 

The  increase  in  local  registration  MSE  is  shown  in  Figure 
5-3. 


The  false  acquisition  probability  as  a  function  of 
tiie  quality  measure  Q  is  seen  in  Figure  4-4.  The  reduction 
in  correlation  peak  value  effectively  reduces  the  signal-to- 
noise  ratio  at  the  correlator  output.  For  Gaussian  auto¬ 
correlation  functions  we  can  obtain  a  correction  factor  k  ^ 
so  that  the  adjusted  quality  measure  Q '  is  given  by 


where  'is  shown  in  Figure  5-2  and  given  by 

-a 

2 

!  *. 

k 9  =  ---  ■  — —  e  ^  d  x  .  (5-37) 

>  2  t:  d  .2- 

0 

The  resulting  increase  in  false  acquisition  probability  de¬ 
fends  on  the  magnitude  of  0  as  seen  in  rigure  5-4. 

The  Effect  of  Match  Window 
Size  on  Correlation  Accuracy 

in  tne  acsence  of  relief  distortion,  the  correlation 
accuracy  is  expected  to  improve  with  increasing  window  size. 
~nis  behavior  is  predicted  by  the  Cramer-Rao  bound  as  well 
as  0  - .  since  0  increases  with  match  window  extent.  In  the 

r  M 

presence  of  terrain  relief,  however,  the  distortion  measure 
varies  d i rec t 1 y  with  match  window  size.  The  result  is  that 
for  d  >  2  ,  t.nere  is  an  optimum  window  size,  N  o  ^  ^  ,  which  will 
minimize  the  correlation  error  in  some  sense.  The  optimum 
value  of  'I,  however,  depends  on  the  criterion  to  be  minimized 
and  there  is  no  general  procedure  for  selecting  the  "best" 
criterion.  Obvious  candidate  criteria  include  (a)  the  mini¬ 
mization  of  P - ,  or  ( b )  the  minimization  of  M  S  E  assumina  cor - 
rest  acquisition.  These  criteria  have  been  examined  by 
M o  s  t  a  f  a  v i  ana  Smith  i  1  9  7  3  a  ,  1978b)  assuming  a  Gaussian  auto¬ 
correlation  function.  They  show  that  the  optimum  value  of 
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is  s  I  i  g  n  1 1  y  larger  ( -  3  0  f; )  than  under  criterion  (b).  In 

either  case  N  .  decreases  with  increased  distortion  as  one 
opt 

would  expect.  Kostafavi  [1379]  has  also  considered  the  opti¬ 
mization  of  window  functions  for  image  correlation.  ~nese 
results  we*'?  derived  uncer  t n e  assumption  that  tne  affine 
transformation  describes  the  distortion  globally,  i.e.,  the 
geometric  distortion  is  independent  of  window  size  and  shape. 
In  stereo-compilation  systems,  however,  the  relief  distortion 
is  location  dependent  and  as  the  window  size  increases  the 
distortion  becomes  more  complex;  for  large  match  windows,  the 
distortion  ^unction  h ( x )  =  ax  may  not  be  valid.  Most  present 
S ter eo- comp i 1 ers  operate  with  a  fixed  match  window  size  and 
perform  spatial  warping  as  described  earlier  to  adjust  for 
local  terrain  variations.  As  stated  previously,  it  would  be 
useful  to  be  able  to  obtain  distortion  information  prior  to 
the  compilation  process.  In  the  following  section,  an  image 
overlay  quality  measure  is  developed  by  considering  the  ef¬ 
fect  of  relief  distortion  on  the  arithmetic  difference  be¬ 
tween  two  registered  images. 

E_s  jb  .  nation  of  Relief  Distortion 

We  will  assume  that  within  sufficiently  small  image 
regions,  that  the  relief  distortion  is  modelled  by  (5-12). 
Repeating  tne  earlier  result. 
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a  =  i  - 


(5-33) 


wnere  H  ^ 

-  e  ( X )  .  T 

he  quantities  ; 

r.  and  H  ,  can  be  viewed 
0 

as  random  ,  a  i 

■'  i  a  b  1  e  s  w  h  i 

:  r  d  e  p  e  n  d  0  r.  t 

ne  match  window  loca- 

t i 0  n .  Since 

H  >  >  e  (  X  )  , 

we  will  let 

hQ  =  H  -  E ( e ( X )  )  (5-39) 

arc  ignore  the  e  f  *  e  c  t  of  variations  in  elevation  on  the 
value  of  a.  From  (5-38),  it  fellows  that 


EU  -  1) 


8  ,2.,  2 


(5-40) 


But  m(X)  =  so  (5-40)  becomes 


?  ! 

9  a  ?  R  (  I  )  I 

E  (a  -  1  ;  =  (tt-/  (  -  - m— 

h0  dT2  !x  =  0 


(5-41) 


where  P.  is  the  elevation  profile  autocorrelation  function. 

As  we  have  seen,  the  difference  between  the  stereo¬ 
pair  images  is  the  result  of  topographic  relief  and  noise 
sources.  Letting  g  .,  (  x  ^  )  and  g  2  (  x  2  )  represent  the  underlying 
image  intensity  profiles,  we  have  seen  that  if  x,  and  x  2  are 
conjugates  t  n e  n  ,  in  general, 

9,  (x]  )  =  g2(x2)  ( 5-42a  ) 


9  ;  (  x  ]  +  - )  t  g  2  (  x  2  +  j )  .  (  5  -  4  2  b  ) 

Lew  suppose  we  select  a  region  of  imagery,  ( c on s i de r a b 1 y 
larger  tear,  match  window  size),  and  register  with  the 
otner  member  c  f  tne  stereo-pair.  The  value  of  parallax  at 
r  e g  ■'  s  t  r a  t  i  0 n  ,  •  ,  provides  us  with  the  mean  elevation, 
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v 


_  =  E  (  e  (  X  )  )  ,  over  the  selected  region  and  defines  a  region 

S„  wnich  is  most  similar  to  S -j  .  If  the  elevation  in  the 
region  is  constant,  then  assjming  the  registration  is  correct. 


the 

d i f ference  between 

e 1 emen  ts 

of  S.|  and  S  ^  is 

given  by 

S  2 ( x  £  )  -  si  Cx]  )  = 

g2(x2)  + 

-  9  i  (  ^  i  y 

O  -j  (  X  -j  ) 

* 

= 

h  2  v  x  2  ^  - 

n i  (x-,  ) 

(5-43) 

t 

a  e  f  i  n  e 

i 

2 

cd 

E($2(x2) 

-  S  ^  (  x  y  )  )  " 

(5-44) 

f  2 

where  -  .  is  the  differ 
a 

ence  variance.  From  (5-43), 

i 

2  ..  , 
:d  "  u 

(  n  2  (  x  2  )  - 

,  ,  ,2  ,2 
n  1  x  1  =  2:n 

(5-45) 

I  r, 

the  presence  of  topographic 

relief,  however , 

the  overlay 

i  3 

less  accurate  and  ; 

2  >  2 2  . 
a  n 

For  the  mean  elevation  u  , 

t  'n  e 

parallax  is  given 

by 

83 

(5-46) 

:  = 

"e 

a  n  d 

the  coordinate  transformation  that  results 

in  correct 

r  e  g 

i s  t  r  a  t i o  n  is  given 

by 

BE 

(5-47) 

X  /y  X 

1  H  -  u 

=  x]  -  5 
e 

r  e 

./  e  now  define 

n  (  x  1  )  = 

S?(x  )j 

L  c  'X 

2 

-  S  (x  ) 

=  xr; 

(5-43) 

the 

n  -2  --  E  (  n  2  (x  .  A 

n  express 

2 

ion  for  that 

d 

includes  the 

r  e  1 

i  e  f  distortion  can 

be  derive 

d  by  considering 

the  .geometry 

of 

Figure  5  -  5 .  If  at 

point  X  i 

n  the  object  soace,  e  ( X )  =  , 

t  r  e 

o  g  1  (  x  ]  )  =  g  ?  (  x  2 ) . 

If  e  (  X  ) 

f  ..  ,  then  as  a 

first  order 

<T>  ro 


?  on 


appro  X  T.  3 


9  2  (  x  ;  =  g  1  (  x 


where  A  is  the  resulting  a i f  f e  r e  n  c  e  in  ground  position  of 
the  r.  r  e  -  i  m.  a  g  e  s  c  f  g  ^  (  x  ,  )  and  ( x ,, )  due  to  the  change  in 
e ( X )  .  if  we  assume  that  e  ( X )  is  a  constant  over  the  interva 
A  ,  then  t  y  similar  t  r  i  a  n  g 1 e  s  , 


A  =  [ e ( X )  -  ue] 


{ 5-50) 


Substituting  (5-50}  into  (5-4S)  and  (5-49}  resul 


t  s  in 


=  |-!*(:<)[e(X)  -  uQ]  +  n 2 ( x 2 )  "  ni^xP 


A*  (Jj>  E(r(X))2E(e(X))  ue)2  +  2;2 


( 5-52) 


where  the  assumption  is  made  that  the  terrain  elevation  and 
terrain  reflectance  processes  are  independent.  Taking  the 
expectations  in  (5-52),  we  have 


2  a  2  d  2  R  r  (  t  )  | 

rC  =  (  £_  )  !  .  _ L 

a  Ho  v  dT2  ! 


,T  =  0 


)  ,2  +  ,,2 

"  e  n 


(5-53) 


is  the  variance  of  the  elevation  data  within  the 
region  encloses  by  3  .  .  Since  the  photograph  intensity  pat¬ 
tern  is  the  image  of  the  ground  pattern,  it  follows  that 


•  d 2 R , ( i )  d2R  (t) 

-  — -  K— 9  —  -  KR~(t) 
do  ^  do  3 


( 5- 54  ) 


where  .<  is  a  constant  which  accounts  for  the  change  in  scale 
and  units  from  object  to  image  space.  Equations  (5-53)  and 
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(5-5-)  can  ~  e  combined  to  yield 


^  il0 

2  \  /  0 , 


d  '  <i:n!  '  BJ  K  P. "  '  (  OT 

r\  '  4 


( 5- 55  ) 


'he  quantity  :  can  be  estimated  by  computing 


2  _  1 
d  *  M 


srs2 


( s 2 ( x 2 ;  -  s, (x1 ) ) 


( 5- 56  ) 


where  M  is  the  total  numoer  of  elements  in  2,.  If  we  now 

I 

assume  that  the  elevation  bandwidth,  £  ,  is  known  and  as¬ 
sumed  to  be  constant  over  the  ensemble  of  images,  then  from 
the  moment  theorem 


d  k  (t) 
e v  ' 


-  (2-0  Ve2 

e  e 


dx  ] t  =  0 

Substituting  (5-57)  and  (5-55)  into  (5-41)  results  in 

•  1 


E 


(a  -  l)2  3  (2-)2S2(zl  -  2o2 


n  '  K““R  '  '  (  0 ) 


(5-57 


(  5  -  58 ; 


Equation  (5-53)  represents  an  estimate  of  the  fluctuation  of 
Ore  distortion  param, eter  a  measured  ever  the  terrain  region 
v i e  w  e  d  t  h  r  o  u  c  h  section  S  ^ . 

_  p 

iO  summarize,  the  procedure  for  estimating  E(a  -  1) 
is  as  follows: 


(  1  )  Select  region  S  ^ 

( 2 )  Register  5  j  with  the  other  image  to 

define  3  ,  and  •: 

2  'J  e 

(  3)  3  cm  o  u  t  c-  the  sample  difference  var  i  - 

-  2 

a  n  c  e  ,  ;  . 


\ 
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(4)  Using  either  S  ^  or  S  9 ,  compute 

d2R  (-)|  2  7  2 

RI'(O)  = - V-i  =  (2TT)2  fZG(f)df  . 

3  d-/  t  =  o  J  g 

'  -  oc 

(5)  Compute  E  ( a  -  l)2  as  in  (5-53). 

The  assumption  that  £  p  is  known  is  certainly  an  oversimplifi¬ 
cation  since  I  is  also  a  random  variable  that  is  location 

P 

dependent.  £g  cannot  be  determined  without  performing  tne 
stereo -com Dilation  process.  In  its  place,  then,  the  best  we 
can  do  is  to  use  £  g  =  E ( £  )  in  (5-53)  where  the  ensemble  mean 
is  estimated  over  an  ensemble  of  known  terrain  conditions. 

\ o t e  also  that  if  there  are  other  differences  between  the 
images  such  as  a  reflection  from  an  object  appearing  bright 
in  an  image  (due  to  aspect  angle)  and  dark  in  the  other  image, 
the  computation  in  (5-53)  will  overestimate  the  variance  of 
the  distribution  of  a.  In  these  situations,  Equation  (5-53) 
may,  however,  find  utility  as  an  overlay  quality  measure 
since  any  factors  that  reduce  the  overlay  quality  are  also 
expecteu  to  degrade  the  correlation  performance.  We  thus 
refer  to  c2  =  E(i  -  l)2  computed  as  in  (5-58)  as  the  "effec¬ 
tive1'  relief  distortion. 

7h e  Effect  of  Distorti on  on  Estimated 
Mean  Square  Correlation  Error 

Assuming  that  the  distribution  of  a ,  p  ( a )  ,  were 
known,  if  is  p o s s i b 1 e  to  adjust  (at  least  theoretically)  the 
e  s  t  i  m  ate  o  f  M  S  E  f 


roin  equations  (4-2)  and  (4-100)  to  (4-1C2) 
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where  MSE(a)  is  computed  from  (4-3)  with 

*  L  u(x,n,£,a) 

P,(a)  =  Z  [  Gaus(y)dy  Gaus(x)dx  .  (5-61) 

L  J  n=-L  J 

-00  -CO 

n= 

In  practice,  however,  it  is  extemely  unlikely  that 
image  data  will  meet  all  the  requirements  leading  to  (5-61) 
(namely  Gaussian,  stationary  data  with  uncorrelated  correlator 
top  outputs).  However,  (5-59)  provides  a  means  of  adjusting 
tne  quality  measure  Q  should  the  distortion  be  deemed  a  poten¬ 
tial  source  of  difficulty.  This  problem  is  discussed  further 
in  the  following  chapter  after  observing  the  behavior  of  cor¬ 
relator  performance  in  a  simulation  of  relief  distortion. 


CHAPTER  6 


CORRELATION  ERROR  PREDICTION  - 
EXPERIMENTAL  RESULTS 

In  the  previous  chapters,  an  attempt  has  been  made  to 
develop  the  theoretical  framework  on  which  to  base  practical 
image  pre-processing  schemes.  In  this  chapter,  we  consider 
the  prob.lem  of  implementing  image  processing  algorithms 
within  such  a  framework.  Although  the  resulting  complexity 
and  throughput  rate  of  the  pre-processor  are  of  some  concern, 
we  will  oostpone  discussion  of  this  topic  until  the  behavior 
of  a  variety  of  processing  schemes  has  been  observed.  To 
simplify  the  processing,  however,  we  will  restrict  our  at¬ 
tention  to  procedures  which  create  "quality  maps"  based  on 
only  one  image  of  the  stereo-pair.  This  eliminates  any 
scheme  that  requires  the  computation  of  cross  spectra  or, 
equivalently,  cross  correlation  as,  for  example,  in  (3-29). 
Instead,  we  will  assume  the  model  of  (3-7)  with  a  =  1.  The 
quality  measures  developed  in  previous  sections  fall  into 
three  categories:  (a)  those  based  on  local  registration 
accuracy,  ( b )  those  based  on  false  acquisition  probability, 
and  (c)  those  based  on  the  detection  of  input  data  no  n  - 
stationarity.  In  category  (a)  we  will  look  specifically  at 
the  Cramer-Rao  bound  expression  (Equation  (3-3)),  since  it 
is  independent  of  the  particular  correlation  algorithm.  In 
category  (b),  we  will  be  concerned  with  the  quality  measure 
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Q  in  its  various  forms  whicn  depend  on  the  parameter  in 
(4-97).  Since  the  measure  0 -j  in  (4-99)  depends  on  the  signal 
variance  which  is  a  measure  of  local  image  contrast  or  tex¬ 
ture,  we  will  also  consider  some  nonparametric  local  statis¬ 
tics  such  a?  contrast  modulation  and  median  absolute  devia¬ 
tion.  These  quantities  will  be  defined  and  discussed  in  de¬ 
tail  in  a  later  section.  In  category  (c)  we  will  confine 
the  discussion  to  the  variance  gradient  as  discussed  in 
Chapter  4 . 

The  objective  of  the  experiments  in  the  following 
sections  is  to  observe  the  behavior  of  the  quality  measures 
and  determine  the  decree  to  which  the  quality  measures  can 
be  used  to  detect  errcr-prone  regions  of  imagery.  Before 
proceeding,  however,  it  is  necessary  to  formulate  a  defini¬ 
tion  of  "errcr-prone'1  and  to  establish  appropriate  detection 
criteria.  To  this  end,  consider  the  stereo  - images  4  and  3 
and  the  two  ensembles  of  "  no  i  se-  images"  {-, }  and  [  z  )  .  We 
assume  the  n o i s e - i m a g e s  are  statistically  independent.  We 
form  a  noisy  stereo-pair  A  and  3  +  ;.|  ,  perform  the 

stereo -compilation  process  and  generate  an  error  map,  e^, 
as  in  Figures  2-9  and  2-11.  If  this  procedure  is  then  re¬ 
peated  for  ^  and  i  =  1  ,  .  .  .  ,  id  we  can  generate  an  average 

squared  error  map, 


2 

e 


1 

N 


(6-1  ) 
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A  point  a.j  in  A  can  be  defined  as  "error-prone"  if  the 

associated  error  point,  e..,  is  such  that  e..  >  t  where  t 

i  j  i  j  c  c 

is  some  error  threshold.  This  definition  is  appropriate  for 

the  quality  measures  which  fall  into  categories  (a)  and  (b) 

above.  For  category  (c),  however,  we  note  from  Figure  2-9 

that  the  covariance  processor  produces  significant  errors  in 

the  absence  of  additive  noise,  presumably  due  to  the  presence 

cf  edges  as  discussed  in  Chapter  4.  For  this  particular  case, 

an  error-prone  point  will  simply  be  one  that  corresDonds  to 

an  error  value  that  is  greater  than  t  . 

There  are  a  variety  of  methods  for  ascertaining  the 

degree  to  which  a  particular  quality  measure  "follows"  the 

error  behavior.  Once  an  error  threshold  t  has  been  estab- 

e 

lished,  the  detection  criterion  reduces  to  a  simple  binary 
hypothesis  test  with  hypotheses  the  point  is  not  error 

prone  and  H ^  :  the  point  is  error  prone.  For  a  particular 
measure  0,  we  can  thus  select  a  quality  threshold  tg  and 
comoute  the  usual  detection  statistics.  Although  the  de¬ 
cision  regions  will  depend  on  the  particular  Q  chosen  we 
will  assume  for  purposes  of  illustration  that  the  larger  Q 
is,  the  less  susceptible  the  image  point  should  be  to  correla¬ 
tion  error  (this  is  reversed  for  the  Cramer -Rao  bound).  With 

this  convention  we  can  compute  the  fraction  of  hits  (H:  e.. 

^  J 

>  t  and  Q  <  tn),  misses  (M:  e..  >  t  and  Q  >  tn),  false 

c  Q  i  j  e  Q 

alarms  (FA:  e.  .  <  t  and  Q  <  tn),  and  correct  rejections 
i  j  e  Q 

(CR:  e . .  <  t  and  0  >  tn).  Under  this  convention, 

i  j  e  Q 
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H  +  M  =  1  and  FA  +  CP.  =  1.  Generation  of  this  data  allows 
the  construction  of  ROC  (receiver  operating  characteristic) 
curves  which  are  plots  of  the  detection  frequency  (P^), 

/ersus  the  false  alarm  frequency  (Pp^). 

It  is  also  instructive  to  compute  the  histograms  of 
the  duality  measures  conditioned  on  eacn  of  the  two  hypotheses. 
Since  it  is  desirable  that  these  distributions  be  separated, 
a  measure  of  the  degree  of  separation  is  given  by 


EtQ;H0>  -  E { Q ! H 1  } 
=Q:H0  *  V.H, 


(6-2) 


2  2  ? 

where  -  n  H  .  =  E ( Q  j H  .  )  -  E  ”  ( Q  :  H  .  )  .  The  larger  c  q  is,  the 
less  sensitive  the  detector  will  be  to  changes  in  the  threshold 
levels  t q  in  the  vicinity  of  the  optimal  (say  in  the  Sayes 
sense)  threshold.  This  is  desirable  since  the  choice  of 
threshold  level  in  a  real  operating  pre-prccessor  must  be 
based  on  simulation  results  (and  experience)  and  for  a  par¬ 
ticular  image,  is  unlikely  to  be  optimal. 

We  shall  proceed  by  first  considering  the  covariance 
processor  errors  produced  by  edges.  We  will  then  restrict 
our  attention  to  the  normalized  covariance  processor  when 
we  consider  the  quality  measures  in  categories  (a)  and  (b). 
Although  the  false  acquisition  quality  measures  were  derived 
for  the  covariance  processor  we  recall  that  normalized  covar¬ 


iance  simply  forces  stationarity  on  the  correlation  process. 


There  is  no  reason  to  believe  that  the  error  behavior  of 
covariance  and  normalized  covariance  are  significantly 
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different  under  conditions  of  stationarity.  In  fact,  in 
Chapter  2,  it  was  shown  that  they  are  equivalent  processors 
under  conditions  of  stationarity. 

Finally,  we  will  present  some  simulations  designed 
to  assess  the  correlation  behavior  in  the  presence  of  distor¬ 
tion  modelled  by  h(x  =  2x  and  tc  determine  the  utility  of 
the  distortion  measure  d  discussed  in  Chapter  5. 

In out-Data  Non-Sta t i o na r i ty 

The  synthetic  image  shown  in  Figure  2 - 4 b  was  passed 
through  the  orocessor  shown  in  Figure  4-8.  The  variance 
estimator  was  implemented  by  computing  the  sample  variance 
within  a  window  having  the  same  dimensions  as  the  match 
window  (7  pixels/ line  x  3  lines,  etc.).  Since  the  similarity 
search  is  along  epipolar  lines,  the  gradient  operator  com- 
outes  the  horizontal  gradient  only  using  the  operator  shown 
in  Figure  6-1.  The  number  of  elements  in  the  gradient 
operator  is  chosen  to  be  the  same  as  the  number  of  pixels/ 
line  in  the  search  window  since,  for  each  match  window  lo¬ 
cation,  we  wish  to  detect  the  presence  of  any  portions  of 
the  search,  window  displaying  rapidly  changing  energy.  We 
assume  that  the  match  window  is  centered  in  the  search 
window,  or  more  precisely,  that  the  correct  subimage  is 
centered  in  the  search  window.  This  is  a  reasonable  approx¬ 
imation  for  a  tracking  correlator  since  the  search  is  carried 
out  in  a  region  that  is  symmetric  about  the  predicted  correct 


sub-image  location.  For  a  non-tracking  correlator,  the  search 
window  is  usually  determined  by  the  expected  extremes  in 
elevation.  Thus  a  match  window  corresponding  to  an  elevation 
that  is  near  one  of  the  extremes  will  result  in  a  correct 
sub  image  which  is  located  near  an  extreme  of  the  search  win¬ 
dow. 

The  covariance  error  maps  from  Chapter  2  and  the  corre¬ 
sponding  variance  horizontal  gradient  maps  are  shown  in  Fig¬ 
ure  6-2  for  m  a  t  c  i  window  sizes  7  x  3  and  15  x  3.  The  dis¬ 
tributions  of  errors  for  these  models  is  nearly  symmetical 
about  zero.  For  this  reason,  the  ROC  curves  in  Figure  6-3 
were  formed  by  taking  the  absolute  value  of  the  data  in 
Figure  6-2  and  altering  the  gradient  threshold  t^  for  a 
given  error  threshold  t  =  1.5.^  The  distribution  separa¬ 
tion  as  a  function  of  tg  is  shown  in  Figure  6-4.  The  stair¬ 
case  appearance  results  from  the  fact  that  the  errors  in 
Figure  6  -  2  a  are  i nteger-va i ued  so  any  threshold  between  n  and 
n  +  1  will  result  in  the  same  separation. 

The  regions  detected  by  three  different  thresholds 
are  shown  in  Figure  6-5  where  the  statistics  are  color  coded. 
The  point  at  which  the  false  alarm  rate  becomes  jnacceptable 
depends  on  the  spatial  relationship  between  the  hit  (or  detec¬ 
tion)  locations  and  the  false  a  1  arm  locations;  false  alarms 
appearing  in  close  proximity  to  hits  can  contribute,  in  this 

^ The  separate  ROC  curves  for  positive  and  negative  valued 
errors  are  not  significantly  different  from  Figure  6-3. 


* —  *  elements  =  4  pixels/line  in  search  window  — 
Figure  6-1.  Horizontal  Gradient  Operator 
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author  *  s  view,  to  acceotabl  e  detection  .  k'nfortuna  tely  ,  the 
to  i  n  t  at  which  false  si  arts  become  excessive  is  somewhat 
subjective. 

Although  the  overall  structure  o  "  the  variance  g  r  a  d  - 
iert  shows  similarities  to  the  covariance  error  nap,  it  is 
clear  that  there  are  errors  produced  by  other  mechanisms 
w n  i  c  h  ,  in  effect,  degrade  the  detection  s  t  a s i s i i c  s .  Also, 
since  the  simulation  wr.s  performed  using  a  non- tracking  cor¬ 
ral  a  tor ,  some  of  the  predicted  error  locations  are  not  seen 
on  the  actual  error  map  (for  example,  the  white  and  black 
parallel  features  in  Figure  6-2 (b)  corresponding  to  the  road 
as  the  right-hand  side  of  Figure  2  -  4  ( b )  .  This  is  due  to  the 
fact  that  the  corresponding  elevation  is  near  the  lower  ex¬ 
treme  and  she  non- tracking  search  window  does  not  overlap  the 
f  e  a t ure  (road)  resulting  in  the  large  variance  gradient) . 
r  u  r  t  k  e  rm  are,  tne  presence  of  an  edge  does  not  gua  a  r.  see  that 
; :  f''o  1  a  s  i  c  n  errors  w,:’l  occur;  they  are  only  more  'ik-ely  at 
t  ■  i e 3  1  o c  a  t  i  ons  . 


’red  i_c_t_i  o  n_  of  Corr  ej  a  t  i_o  n  : 
* o  r  Stationary  J a  t a 


7  n  e  .n rma  i  i  z  a  d  c  o  v  a  r  i  a  nee  p  r  ocess  sr  :  •• ::  ;  ;  e  :  ;  :  a  t :  c  n  - 

a  r  i  t  /  b  y  n  c  rm  a  1  i  a  i  r  j  t  r  e  i  m  a  g  e  d  a  t  a  s  e  a  u  a  r.  c  e  s  .  2  y  -a "  ;s  1  ;  y  i  n  g 

tne  n o  r m a  1  i  c e  !  c s  v a  r  i  a  n c  e  ;; r o c s s o r  ,  t r. e  or  e  s  e nee  o f  e dges 
c  a  .n  s  e  i  gr.or  d  a  s  a  s  o  u  r  c  e  ;•  c  ;  a  r  r  e  let;  o  n  e  r  r o  r  and  w e  need 

e  •;  o  n  s  e  r  n  ;•  i  o  r  1  .  -  f  1  seal  r  :  i  s  t  r  a  t  i  o  n  a  n  d  f  a  '  s  e  acquisi¬ 

tion  e  r  r  o  r  s  .  k  i  t  h  o  u  t  c r  v  i  n  :  e  a  c  h  c  o  r  r  elation  function. 


r.  c  w  ever,  it  is  •  j  i  f  f  i  c  u !  t  to  classify  or.  error  as  doe  so  1  o  c  a  ' 
r  •;  :  i  :  r  a  t  i  o  r.  or  false  acquisition.  ~c  r  this  re  a  s  o  r  ,  we  will 
to  concerned  o n  1  y  w i  t h  t h o  d e g r e 2  to  which  the  i ;r. age  quality 
0  j  c  u  r °  s  o  r i-j diet  "the  error  b 0  h c  v  i  0  r 


Figure  5-6  shows  thee  r  r  c  ’"maps  ■'  .’•*  S  E  }  genera 


I  n 


1  o n t e  Carlo  simulation  as  described  earlier  for  too  match 


w  ■,  n  a  o  w  s  s 


o  anc 


v: i  t h  an  overall  input  s  i  o  n  a  1  - 


*  o  -  n  o  i  s  e  ratio  of  2  C  d  3  .  As  in  Chapter  2  ,  the  i.xag  i  n  tensity 
ass  c c i a  t  e d  w i t  h  a  particular  point  in  Fi cure  6-6  is  1 i nearly 
’"elated  to  t h e  mean-square-error  associated  with  that  point. 

r.  ore  e  "  to  observe  the  s  e  n  s  i  t  i  v  it;/  of  the  various  quality 
measures  to  noisy  source  data,  the  quality  measures  were  co cl¬ 
outed  over  both  clean  and  noise  contaminated  imagery. 

C o r.p u  t  a  t  i  o  n  of  the  C u a  l_i_  tv  '■! e a  s u  r e s 

A  list  of  the  -easures  on pi oyed  and  their  definitions 
i  s  given  in.  Table  6-1.  Computation  of  the  false  acquisition 
and  contrast  measures  are  straightforward.  The  vector  r 
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ihe  computation  of  the  Cratner-Rao  bound  expression 
requires  the  estimation  of  the  test  window  power  spectrum. 
Since  the  correlation  search  is  one-dimensional  ( e  p i p  o  1  a  r 
lines},  CRB  is  computed  by  averaging  these  quantities  over 
tre  lines  of  the  test  window.  Since  the  number  of  data 
samples  pe«-  line  (7  or  15)  is  small,  the  data  is  first  mul¬ 
ti  c  lies  by  an  approximation  of  the  Kaiser  winaow  to  smooth 
-he  spectral  estimate  [Harri s  1973],  The  Kaiser  window 
a p p r : x i m a t i o n  is  given  by 

w(n)  =  -  a ,  ccs(— -)  +  a2  ccs(|-2n) 

P  o 

'  a->  cc  s  ( |--3n )  :  n  =  0,1 . N  -1 

p  P 


w  here 


aQ  =  0.40243 
a 1  =  0.49804 
a  ^  =  0.09831 


a 3  -  0.00122 

1  n  :  i  the  number  of  pixels  per  test  window  line.  ,41- 

t.nsurn  otnor  windows  could  certainly  be  employed,  it  was 
e  0 r' f r- t  computation  of  CRB  without  any  windowing  led  to 
■- t  r  c  nc.  1  y  erratic  results. 

■■■  "tor  windowing  the  test  line,  an  estimate  of  the 
SOwc'  spectrum  is  obtained  by  computing  ther  peri odogram 


1975 


Since  t  h  i 


procedure  provides 


CO 


V 


D  6 


an  estimate  of  G  (f)  insteaa  of  G  (f),  we  form  the  estimate 
r  a 


G  (f) 
r '  ' 


f  G  (  f ) 

r 


Vf> 


(6-4) 


j^G  otherwise 

If  it  should  happen  that  G  (f)  -  0  for  all  samples  over  the 

9 

per  - cdogram ,  then  CPB  is  set  to  an  arbitrary  large  value. 

’.■.hen  computing  the  Cramer-R so  bound  on  clean  imagery,  G  ( f ) 

2 

can  ce  measured  directly  and  j  is  based  on  knowledge  of  the 

n 

associated  noisy  image  statistics. 


Error  Prediction  Performance 

Figure  5-7  shows  ROC  curves  for  the  measurements  made 
on  clean  imagery.  The  corresponding  separation  measure  is 
given  in  Figure  6-3.  The  ROC  curves  and  separation  measure 
for  tne  noise  contaminated  case  are  shown  in  Figure  6-9  and 
5-10  respectively.  Comparisons  of  measurements  made  on 
clean  vs.  noisy  data  for  selected  features  are  shown  in 
Figure  5-11  which  displays  the  percent  reduction  in  detec¬ 
tion  frequency  due  to  the  presence  of  noise.  Although  these 
results  are  based  only  on  the  synthetic  images  of  Figure  2-4, 
tne  following  observations  are  noted: 

( 1  )  The  Cramer  - ?.ao  measure  provides  reasonably 
good  detection  rates  only  for  measurements 
based  on  the  15  x  3  window  over  imagery 
that  is  free  of  noise  contamination.  The 


poor  performance  for 


the  7  x  3  window  case 


o  o 

TOO.  “%  ' 


is  a  1 1  r i b  u 

t  p  rj  ^  Q  *■  c. 

moor  spectral  es 

t  i  rn  a  t 

octainable 

o  n  such  3  rr. 

all  data  records 

Since  t  h o 

C  r  a  m  e  r  -  F:  a  o 

reas jr o  w eights 

the 

i m age  oowe 

r  spectrum 

by  t r o  square  o r 

the 

freauency, 

it  is  not 

surprising  that 

it  is 

sensitive  to  the  presence  c *  white  noise. 

The  R 0  C  curves  for  the  variance  and  the 
r'alse  acquisition  quality  measures,  Q,  are 
identical.  This  results  from  the  assumption 
that  the  noise  oower  is  constant  with  sub- 
image  location  and  the  one-to-one  relation¬ 
ship  between  S‘,'R  and  0  specified  by  Equa¬ 
tion  (4-97).  These  quantities  differ  greatly, 
however,  in  separation. 

"he  low  separation  for  low  error  thresholds 
seen  in  Figure  5-3  and  6-10  are  due  primarily 
to  the  one-pixel  errors  which  are  independent 
of  the  local  image  statistics  when  the  error 
threshold  exceeds  1.0,  these  are,  in  effect, 
considered  as  non-errors  and  the  separation 
m e  a  s  u r  e  r  e  spends  accordingly. 

For  a  given  quality  measure ,  an  increase  in 
separation  implies  improved  detection  (  i  .  e  .  , 
the  hit/ false  alarm  ratio  increases'.  With 
f o w  exceptions,  the  separation  is  essentially 
independent  of  error  threshold  for  thresholds 
greater  than  1.5.  When  comparing  two  features 


V 


however,  t  n e  fact  that  one  co  a  tare  has 
creator  seoara  ti on  than  another  feature 
does  not  r. eo e 3 s a r  i  1  y  i m p  1  y  improved  de¬ 
tection  behavior  ■'  e .  c  .  ,  MAD  with  7  x  3 


window) . 

"he  v a  r  i  3 

nee,  0  , 

and  the  v a  r i 0  u  s 

c  0 

ntras: 

r,  easures 

p  r  0  v  i  d  e 

r  e  r.  a  r  k  ably  s  i  m  i 

i  a  r 

ly  P.OC 

b  e  h  a  v  i  0  r 

3  n  d  a  1 1 

are  relatively 

i  n  s 

e  n  s  i  t  i  v  e 

to  noise. 

{  6  1  The  ROC  curves  of  Figure  £  - 1 0  result  from 
measurements  made  on  a  single  noise  con¬ 
taminated  image.  A  Monte-Carlo  approach 
similar  to  the  procedure  used  to  define 
"error-crone  points"  could  be  employed  but 
is  c  0  rr.  o  u  t  a  t  i  0  r.  a  1  1  y  prohibitive.  Further 
e  x  p  e  r i m  e  n  t  a  t i 0 n  has  shown,  h  c w  e  v  e  r  ,  t  h  a  t 
the  curves  of  Figure  6-9  are  representative 
a  1  shout  h  s  a  m  .a  v  a  r  i  a  t  i  0  n  a  n  J  i  r,  v  0  r  s  i  0 .0  0  f 
order-  occurs  between  t  r i a  1 s .  One  can  con¬ 
clude  0 n I  ■/  that  the  behavior  of  7 , 0  ,  C  ,  C  , 

r  m 

a  n  b  D  ^  a  r  2  n  e  r  1  y  equivalent  1  in  the  ROC 
s  e n  s  e  }  a  n d  t h a  t  V A  7)  3  e  e m s  to  i  m prove  w i  t h 
w i n d 0 w  s i 2 e  nor e  than  the  0 thor  features 
d  0  . 

'7  ~  r.  e  s  '-carat  i  mi  "easure  car.  be  better  under- 

:  .  :•  c c  :■  y  •;  0  n  s  i  d e  r  i  n  j  F  i  g u  re  5-19.  F  i  cures 
6  - 1  ~ a ,  b  ,  and  :  a  r 0  r e 0 resent a  t  i  1  n s  of  t h e 


1  u 2  1  ■  t y  measure  ?  f  sr  throe  :  1  f ferent  vdue$ 
of  th  o  3:v'an'eter  .  7  a  r  '  r, r  those  f  i  cures 

to  the  s  o 3 a  r a  t  i  o  n  :r, e  a  s  u  r  o  s  ;  n  Figure  5 - c a  , 
it  can  bo  teen  that  i n o re  a  sod  separation  re¬ 
sults  in  a  data  o' stri out  ion  that  is  i  n  - 
:  r  o  a  s  *  r.  c  1  y  t  i  "■  c  d  a  1  ;  that  is,  the  feature- 
data  is  separated  into  twe  classes  w h i'c h  be¬ 
come  acre  distinct  as  t'ne  separation  measure 
i  n  c.  rouses.  T  r.  e  d  c  c  r  ease  in  separation  re  - 
suiting  -rorr.  rca  suremen  t  s  on  noisy  imagery 
can  to  observed  in  Figures  6-1  2d  ,  e ,  a  n  c  f . 
We  also  note  the  t  the  decrease  in  separation 
resulting  fro1'  ~ea  su regents  or,  noisy  images 
increases  with  s e p a r a  t  i  o n  .  -or  exa^pl e  ,  the 
separation  of  the  v  a  r  i  a  n  c  e  r.ap  on  a  iges 

little,  if  any,  but  t h o  seoarati c n  o f  Q 

1 

-'or  •  =  is  considerably  reduced. 

.  i 

~  h  o  sect  ion  o  a  pability  far  the  15  x  3 
window  is  superior  in  all  respects  tc  that 
f o r  the  7  x  2  w i n d o w  .  Since  parameter 
e  <  t  i t  i  o  n  e  .  c  .  ,  variance,  spectra m ) 

•:  :■  n  e  r  a  1  1  /  i p  r  o  v  e  s  w  i  t  h  tne  quantity  of 
d  a  t a ,  i t  i :  r  e  a  s o n able  t o  e <  pec  t  further 
h'tr;  /  .->•’  ent  far  larger  window  sizes.  The 
par  a '•..-.•tors ,  however,  oust  be  re  ore  sen  ta- 
t  i  v e  a  f  t  h q  ••  a  t  c  h  w  i  ndo w  a  t  a  given  1  o  - 
.  1 1 '  o  n  j  n  thus  the  test  window  cannot 
i  n<:  v  s  i  r.  s  i  z e  i  n d  n  f  ini  t e  1  y  . 


1  70 

Tne  ROC  curves  «r;d  se:  a  rat  ion  m  e a  sure  do  not  provide 
a  ccmp  1  o  te  :  e s  c  r  i  p  t  i  c  r.  y  f  t n  e  p e  r  f  o  r r.i h nee  o  f  a  par  :  i  cola  r 
image  featj-~e  *  or  t h  e  s  i  ;np  1  c  reason  that  a  given  *a  ’  se  alarm 
rate  p r  c  v  •  c e s  n o  i  a  forma  ticn  c  o n c e  r n  i  n  g  the  spatial  r e  1  a  t  i  o n  - 
ship  between  the  locations  of  nits  and  false  alarms;  a  false 
a  1  d  r::i  that  is  in  close  p  r  o  x  i  m  i  t  y  to  a  hit  location  is  more 
i  r,  a  i  c  a  t  i  /  e  o  *  t  n  -a  t  r  u  e  error  behavior  than  a  false  alarm  i  r. 
a  region  w.nere  *ew  or'  no  errors  occur.  Because  of  this  spa¬ 
tial  relationship,  the  point  at  which  the  false  alarm  fre¬ 
quency  becomes  excessive  is  difficult  to  define.  As  an  ex¬ 
ample,  the  color-coded  detection  statistics  for  the  variance 
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How 

eve'-,  for  FA  -  .13,  numerous  false  alarms  appear  at  locations 
cistant  from  hit  locations.  This  behavior  is  also  feature 
oeoendent;  i n  Figure  6- : 2b ,  we  see  that  isolate  a  false  alarms 
a  s  o  e  a  r  at  v  e  r  y  low  nit  r  a  t  e  s  1.35)  f  c  r  the  C  r  a  m e  r - R  a  o  feature 
measures  ;  /  noisy  imagery. 

~  n  r  e  s  n  o 1 s  Sensitivity 

r i cure  6-13  implies  cha  t  the  variance,  3 ,  ana  the 
/a  r • ous  contrast  measures  are  relative! y  i nsens i ti  /e  to  tne 
introduction  of  a  a  s i t i v e  noise.  "nese  curves  are  generated 
o y  altering  tne  '‘eai^re  decision  r e g  i  o n s  o y  adjusting  tne 
*  e  a  t  u r e  tnresnolc  level  ana  it  is  important  to  tetermi ne  tne 
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sensitivity  of  the 

threshold  setting  to 

the 

presence  of 

noise.  If  a  given 

feature  provides  a  10 

T  f  a 

lse  alarm  rate 

with  a  t  n r  e  s  h o 1 d  s  e 

1 1  i  n  g 

of  t -  when  m e a s 

u  r  ed 

over  clean 

imagery,  it  is  desi 

ra bl  e 

that  the  change 

i  n 

threshold  r  e  - 

q u i r e d  to  provide  a 

1  Os 

false  alarm  ra  te 

i  n 

t In e  presence  of 

noise  be  m  i  n  i  m a  1  ,  or  equivalently,  we  would  like  the  change 
in  false  alarn  rate  (cr  hit  rate)  for  a  given  threshold  to 
be  insignificant.  For  the  features  which  are  based  on  para¬ 
metric  statistics  (variance,  Q,  CRB)  the  appropriate  threshold 

? 

depends  on  the  accuracy  or  the  noise  power  estimate,  z~  . 

Since  the  estimate  of  the  SNR  incorporates  the  noise  power 
estimate,  the  false  acquisition  based  measures  ( Q )  adjust 
accordingly.  The  variance  measure  could  be  likewise  adjusted. 
As  an  example,  Table  5-2  presents  some  of  the  raw  data  used 
to  generate  the  ROC  curves  of  Figures  6-7b  and  5  -  9  b .  Vie 
note  that  co r  ■  -  0,  0  provides  nearly  equivalent  hit  rates 
(  H )  for  t  n e  given  feature  thresholds  ( T ) .  This  is  not  the 
case,  however,  for  the  adaptive  '•  or  the  variance.  The 
variance,  however,  can  oe  adjusted  by  subtracting  the  noise 
power  (=25)  from  the  threshold  for  the  noise  case  to  pro¬ 
vide  nearly  equivalent  detection  rates. 

The  features  based  on  non-parametric  statistics  (.MAD, 

C  ^  ,  C  ,  C H )  cannot  be  adjusted  so  simply  since  the  increase 
in  feature  value  due  to  the  injection  of  known  noise  power 
is  difficult  to  ascertain  and  is  best  determines  emo i r i ca 1 1  / . 


fj 
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Table  5  -  2  .  ROC  Curve  Raw  Data 


2 

,  \  =  0  Noise  Estimate:  rr  =  2o. 


T 

4.0 

5 . 0 

6.0 

6.0 

10.0 

12.0 

H 

.49 

.  5  1 

.  70 

.33 

.90 

.94 

Clean 

FA 

.007 

.016 

.029 

.067 

.  124 

.200 

H 

.43 

.62 

.70 

.33 

.  90 

.94 

Noise 

FA 

.007 

.018 

.033 

.072 

.  129 

.202 

x  ,  *  =  a  Q  d  p  1 1  V  6  N  0  1  S  6 

Estimate: 

o  =  25. 

n 

T  1.0 

1.25 

1.3  1 

.  5 

2.0 

H  .72 

.83 

.85 

.90 

.97 

FA  .032 

.070 

.079 

.  123 

.  300 

H 

.  5  3 

.  76 

.  79 

.37 

.96 

'i  o  i  s  e 

FA 

.010 

.  044 

.052 

.097 

.272 

7  a  r  i  a  n  c  e 

7  50 

ICO 

125 

1  5  0 

200 

H  .65 

.  33 

.87 

.90 

.94 

•  r*.  .  J  Cl 

.  069 

.  097 

.  1  30 

.  190 

.  7  5 

.32 

r\  -» 
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A  series  of  experiments  was  performed  to  observe  the 
behavior  of  the  correlation  processors  in  the  presence  of 
geometric  distortion  described  by  the  distortion  function 
n(x)  =  •■;<  where  a  is  defined  in  (5-12).  Table  6-3  shows  the 
approximate  value  of  a - 1 •  as  a  function  of  the  effective 
base-height  ratio  S / H  ^  and  the  terrain  slope  m .  The  simula¬ 
tions  were  performed  for  values  of  2  satisfying  jx-lj  <  .2 
as  follows 

(1)  A  "white"  noise  sequence  is  generated 
using  a  random  number  generator. 

(2)  A  "signal"  is  produced  by  filtering  the 
noise  sequence  with  the  low-pass  transfer 
function  e  ^'s  where  f  is  the  normalized 
spatial  frequency  (‘lyquist  folding  fre¬ 
quency  =  .5)  and  s  is  the  "shaoing  fac¬ 
tor"  which  controls  the  spatial  frequency 
content  of  tne  signal;  the  signal  band¬ 
width  increases  with  s . 

(3)  A  sample  sequence,  s,  consistina  of  N 

s 

samples  is  t.nen  selected  and  stored. 

(A)  A  central  portion  of  s  is  distorted  and 
resampled  using  Lagrange  interpolation 
[Stark  1970;  to  producf  a  sequence  r. 

'■  3 )  White  Gaussian  uncorrelated  noise  of  known 
power  is  added  to  each  sequence  independ- 
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Table  6-3. 


'a  -  1;  as  a  Function  or  Base- Height 
and  Terrain  Slope 
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(6)  The  noisy  match  sequence,  r  +  n  ,  is  then 
correlated  with  the  noisy  search  sequence, 

s  +■  n  ,  to  generate  a  correlation  function. 

(7)  Toe  computed  peak  of  the  correlation  function 
is  compared  to  the  correct  peak  location  and 
the  error  is  retained. 

(3)  Steps  3-7  are  reseated  M  times  and  the  re¬ 
sults  averaged  to  yield  MSE  values  for  each 
value  of  the  distortion  parameter. 

(9)  Steps  2-3  are  repeated  for  several  shaping 
factors  and  signal-to-noise  ratios. 

One  product  of  this  procedure  is  a  tv/c  -  d  :  me  n :  i  o  r  a  1 


function  which 

is  similar  in 

nature  to 

tne  so-called 

1  a  p  i  - 

guity  function 

1  encountered 

in  radar  s 

i  g  r.  a  1  analysis. 

~  n  e 

ambiguity  function,  c(t,j),  describes  tne  benavior  c  *  :,ne 
correlation  function  as  the  parameter  •  is  varies.  Figure 
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shows  typica 
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at  tne  correct  p  e  a  <  location.  This  effect  is  negiig'ole, 
however,  for  broaa  correlation  functions  (s  =  .01).  The 


functions  in  Figure  5-14  were  generated  using  a  normalized 
covariance  processor;  the  results  for  unnormalized  covariance 
are  similar  in  appearance. 
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The  mean-square  correlation  errors  associated  with 
t n e  r  - e v i o u s 1 y  described  procedure  are  shown  in  Tables  6 - 4 
and  6-5  for  simple  covariance  and  normalizes  covariance  re¬ 
spectively.  Also  given  are  the  associated  distortion  measure 
defined  by  Equation  (5-27).  Each  entry  in  the  table  corre¬ 
sponds  to  20  repetitions  ( ’•!  =  20)  at  an  input  S  ,N  R  of  3  0  d  3  . 


The 
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energ 
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For 

this  reason. 

the  true  input 

SNR 

tends  to  increase  with  shaping  factor  s.  For  s  >  .1,  the 
errors  are  primarily  due  to  distortion;  for  s  <  .1  the  low 
S'lR  results  in  increased  error  magnitude  which  is  essentially 
independent  of  tine  distortion.  A  comparison  of  tables  5-4 
and  5-5  implies  that  simple  covariance  is  slightly  mere  sen¬ 
sitive  to  distortion  and  degrades  'more  rapidly  with  decreasin 
SNR  than  does  normalized  covariance.  This  data  also  indi¬ 
cates  that  one  can  expect  distortion  produced  errors  for 
values  of  the  distortion  measure  d  exceeding,  say,  2.1.  “bus 
t n e  inequality  in  (5-28;  should  be  satisfied  o j  at  least  a 
factor  of  13  for  this  application  if  distortion  is  to  be 
i c no r ed  . 
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CHAPTER  7 


SUMMARY  AND  CONCLUSIONS 

The  major  contribution  of  this  dissertation  is  the 
development  and  experimental  comparison  of  a  number  of  image 
quality  measures  which,  based  on  the  results  of  Chapter  6, 
have  exhibited  potential  for  predicting  the  locations  of 
"error  prone"  image  data.  In  the  process  of  developing 
these  features,  it  was  possible  to  establish  a  theoretical 
framework  for  the  comparison  of  covariance,  least  squares, 
and  normalized  covariance  as  maximum  likelihood  processors 
(Chapter  2).  Observations  regarding  the  accuracy  of  these 
processors  revealed  that  the  small  match  window  size  and 
image  non-station3ri ty  violate  the  assumptions  leading  to 
expressions  for  theoretical  correlator  accuracy  (Chapter  3) 
with  the  result  that  the  performance,  in  terms  of  MSE,  of 
the  covariance  processor  is  far  worse  than  is  predicted 
theoretically.  Methods  for  detecting  covariance  errors 
associated  with  image  non - s ta t i ona r i ty  have  been  considered 
and  have  met  with  limited  success.  The  covariance  function 
has  been  analyzed  in  detail  to  determine  the  relative  cor¬ 
relation  output  signal-to-noise  ratios  associated  with  sev¬ 
eral  noise  sources  (Chapter  4).  This  information  was  re¬ 
quired  in  order  to  specify  the  parameters  associated  with  the 
false  acquisition  measure  Q.  Finally,  the  effects  of  image 
distortion  have  been  considered  (Chapter  5)  and  we  nave 
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concluded  that  the  prediction  of  distortion  related  errors 
is  best  accomplished  "on-line",  i.e.,  as  the  correlation 
process  is  underway,  although  needed  signal  bandwidth  infor¬ 
mation  could  be  provided  in  a  pre-processing  mode. 

In  this  chapter,  we  provide  a  brief  summary  of  the 
more  important  results  and  indicate,  where  applicable,  the 
implications  regarding  the  physical  implementation  of  the 
various  processing  schemes.  In  Chapter  1,  we  alluded  to  the 
possibility  of  enhancing  imagery  to  make  it  less  susceptible 
to  error  producing  mechanisms.  Vie  will  briefly  consider 
this  topic  here  and  provide  some  experimental  results.  Fi¬ 
nally,  a  few  comments  concerning  the  possible  directions  of 
future  work  in  this  area  will  be  presented. 

Correlation  Algorithms 

In  Chapter  2,  we  showed  that  if  we  assume  the  simple 
image  formation  model, 

r  =  ag  +  nr  ,  (7-1 ) 

then  maximum  likelihood  correlation  processors  could  be  de¬ 
rived,  the  form  of  which  depends  on  the  a-priori  assumptions 
regarding  the  parameter  a.  Covariance,  least-squares,  and 
normalized  covariance  are  the  products  of  this  approach.  If 
the  stationarity  assumption  which  leads  to  covariance  as  the 
ML  processor  is  violated,  disastrous  error  conditions  can 
occur.  If  we  assume  that  the  conditions  (original  stereo¬ 
pair,  geometry,  etc.)  under  which  this  behavior  was  observed 
(see  Figure  2-5)  are  not  pathological,  then  the  capability 
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of  switching  to  a  more  reliable  algorithm  in  the  vicinity 
of  regions  violating  stationarity  is  certainly  desirable. 

The  implementation  of  such  a  switching  capability  was  dis¬ 
cussed  earlier  (Chapter  4).  The  question  arises,  however, 
as  to  which  alternative  algorithm  should  be  selected.  Both 
1  east-squares  and  normalized  covariance  do  not  require  the 
stationarity  assumption.  Least-squares  requires  the  assump¬ 
tion  that  a  =  1  while  normalized  covariance  requires  only 
the  assumption  of  the  image  model  which,  as  discussed  ear¬ 
lier,  is  not  entirely  valid  due  to  the  signal  dependence  of 
the  noise.  Since  these  procedures  are  nearly  equivalent  in 
terms  of  computational  load,  it  would  seem  natural  to  select 
normalized  covariance.  The  correlation  experiments  in  Chap¬ 
ter  2,  however,  seem  to  indicate  that  it  may  be  advantageous 
to  employ  least-squares.  In  order  to  assess  the  advisea- 
bility  of  using  least-squares  as  opposed  to  normalized  co- 
variance,  the  Monte  Carlo  correlation  experiment  was  repeated 
for  least-squares.  The  resulting  overall  MSE  values  for  the 
15x3  window  and  2 Od 8  additive  noise  were  .182  for  least- 
squares  and  .615  for  normalized  covariance  (measure  in  units 
of  pixel  spacing).  Thus  for  the  images  and  parameters  of 
this  simulation,  1 ea s t- squa r es  is  considerably  more  accurate 
than  normalized  covariance,  particularly  since  the  MSE  values 
include  the  unavoidable  contour-related  errors.  We  note 
further  that  the  maximum  MSE  associated  with  any  image  point 
was  20.0  for  1 ea s t- squa res  and  60.0  for  normalized  covariance. 
A  close  look  at  these  error  maps  reveals  that  the  contour- 
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related  errors  are  slightly  more  pronounced  for  the  least- 
squares  processor  which  results  in  the  contours  being  more 
"smeared  out".  The  least-squares  processor,  however,  is 
less  susceptible  to  errors  resulting  from  low  SNR,  at  least 
for  the  source  images  employed  in  this  simulation. 

Since  the  ROC  curves  in  Figures  6-7  and  6-9  are  based 
only  on  the  normalized  covariance,  the  experiment  was  re¬ 
peated  for  the  least-squares  error  map.  Figure  7-1  compares 
the  ROC  curves  for  least-squares  and  norma lized  covariance 
for  the  variance  measure.  The  improvement  in  detection  capa¬ 
bility  for  least-squares,  as  well  as  the  improvement  in  MSE, 
provides  a  strong  argument  in  favor  of  least-squares  as  an 
alternative  algorithm.  If,  however,  the  assumption  that 
a  =  1  is  not  valid,  then  it  would  be  necessary  to  know  the 
point  at  which  normalized  covariance  surpasses  1 ea s t- s q ua r es 
in  performance  (if,  indeed,  such  a  point  exists).  We  have 
not  pursued  this  topic  further. 

The  implementation  of  1 e as t- s qua r es  or  normalized 
covariance  requires  only  a  few  simple  additions  to  Figure 
4-1  in  order  to  compute  the  energy  of  the  search  window  sub¬ 
image.  The  resulting  processor  is  shown  in  Figure  7-2  in 
which  covariance,  least-squares,  and  normalized  covariance 
are  all  shown  as  processor  outputs.  With  such  an  implemen¬ 
tation  there  is  a  small  decrease  in  throughput  rate  and  only 
a  slight  increase  in  complexity.  Because  least-squares  re¬ 
quires  only  a  shift  register  (multiplication  by  2)  and  a 


Figure  7-2.  Correlation  Processors 
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subtraction  circuit,  it  is  the  more  desireable  from  a  hard¬ 
ware  viewpoint.  From  our  observation  regarding  relative 
processor  accuracy,  it  would  seem  that  such  a  minor  increase 
in  complexity  is  well  worth  the  investment. 

Error  Variance 

In  the  analysis  of  local  registration  errors,  we  as¬ 
sumed  that  the  correct  correlation  lobe  is  selected,  and 
furthermore  that  the  image  subsections  are  large  enough  so 
that  the  Fourier  components  are  uncorrel ated .  This  approach 
led  to  the  development  of  the  Cramer-Rao  bound  on  the  accu¬ 
racy  of  any  unbiased  correlation  processor.  An  analysis  of 
the  generalized  correlator  resulted  in  a  generalized  maximum 
likelihood  correlator  (given  by  (2-36)  and  (2-37))  which 
achieves  the  lower  bound  under  aforementioned  assumptions 
plus  stationarity.  Under  conditions  of  high  signal-to- 
noise  ratio,  the  covariance  processor  is  nearly  equivalent 
to  this  "minimum  variance"  processor.  This  equivalence, 
however,  implies  that  the  generalized  ML  processor  will  also 
be  susceptible  to  edge-produced  errors  since  these  edges 
typically  are  associated  with  regions  of  hign  SNR.  Thus, 
while  the  tneory  establishes  relationships  which  aid  in  the 
understanding  of  the  correlation  process,  we  have  found  that 
the  violation  of  assumptions  necessitates  an  empirical  a  p  - 
proacn  regardingspecific  processor  accuracies,  at  least 
for  the  application  at  nand  where  the  small  size  of  the 
match  window  is  a  significant  factor.  On  the  other  hand, 
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when  correlating  large  regions  of  imagery,  the  statistics 
of  the  match  area  are  usually  representative  of  the  statis¬ 
tics  of  the  search  area  and  the  presence  of  small  regions 
with  atypical  statistics  can  be  ignored. 

The  analysis  of  false  acquisition  errors  led  to  the 
development  of  the  image  quality  measure  Q  which  depends  on 
the  local  image  signal-to-noise  ratio.  The  value  of  Q  is 
related  to  the  single-tap  probability  of  false  acquisition. 
The  overall  probability  of  f a  1 s e- a cq u i s i t i on  was  found  to  be 
impossible,  in  general,  to  obtain  analytically  due  to  the 
non-zero  correlator  tap  cross-covariances.  Thus,  unlike  the 
Cramer-Rao  bound,  the  magnitude  of  Q  cannot  be  directly  re¬ 
lated  to  mean-square-error  without  extensive  simulation. 

The  form  of  Q  depends  on  the  assumptions  made  concerning  the 
degree  to  which  self-noise  contributes  to  error  production. 
The  results  of  Chapter  6  have  shown,  however,  that  the  ROC 
curve  associated  with  Q  is  independent  of  this  assumption 
(i.e.,  choice  of  X).  Furthermore,  the  low  apparent  sensi¬ 
tivity  of  the  threshold  of  the  A  =  0  version  of  Q  to  noise 
contamination  makes  this  feature  an  attractive  candidate. 

A  comparison  of  the  range  of  magnitudes  of  the  features 
associated  with  the  three  versions  of  Q  (A  =  0,  A  =  X 

adaptive)  are  shown  in  Figure  7-3.  The  numbers  marked  on 
the  range  designation  bars  indicate  the  hit  rate  associated 
with  tne  particular  value  of  Q.  Since  we  cannot  translate 
the  single-tap  false  acquisition  probability  to  overall 
false  acquisition  probability  it  is  difficult  to  determine 
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which  model  most  accurately  represents  the  process.  One 
advantage  of  the  A  =  0  model  is  that  the  change  in  detection 
rate,  say  from  .5  to  .9,  requires  a  larger  percentage  change 
in  threshold  (Q)  than  for  the  other  models.  Thus  this  mea¬ 
sure  is  least  sensitive  to  errors  in  the  choice  of  Q. 

The  detection  capabilities  of  the  Cramer-?. ao  bound 
and  Q  are  remarkably  similar  for  the  15  x  3  matcn  window, 
no-noise  measurements.  This  similarity  is  not  ;c  surprising 
however,  when  the  nature  of  these  measures  is  considered. 

As  stated  previously,  the  Cramer-Rao  bound  measures  the 
curvature  of  the  autocorrelation  function  at  the  origin  (or 
peak)  whereas  Q  is  directly  related  to  the  peak  magnitude 
of  the  autocorrelation  function.  It  is  not  unreasonable  to 
expect  these  quantities  to  be  highly  correlated  with  the  re¬ 
sulting  coupled  behavior  depicted  in  Figure  7-4;  that  is, 
as  the  oeak  of  the  function  decreases  (c0{C)  <  0,(9)),  the 

L.  I 

autocorrelation  width  increases.  Although  there  are  excep¬ 
tions  to  this  behavior,  they  do  not  appear  to  occur  with 
sufficient  frequency  to  alter  the  detection  capabilities. 

Thus  the  erratic  behavior  of  the  Cramer-Rao  measure  in  the 
presence  of  noise  makes  the  use  of  Q  all  the  more  attractive. 
Furthermore,  since  the  magnitude  of  the  false  acquisition 
errors  tend  to  be  larger  by  their  very  nature  (see  Figure 
1-1)  than  local  registration  errors,  they  are  more  likely  to 
produce  a  "lost"  condition  in  a  tracking  correlator  and, 
therefore,  a  measure  which  is  sensitive  to  image  data  re¬ 
sulting  in  false  acquisitions  is  certainly  preferable. 
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Implementation  of  Image 
Quality  Measures 

The  computation  of  image  features  can  be  performed 
digitally  by  scanning  the  film  in  a  manner  similar  to  the 
scanning  required  by  the  stereo-compiler.  Once  the  sub  image 
of  interest  has  been  digitized,  the  features  listed  in  Table 
6-1  can  be  computed  in  a  straightforward  manner.  The  con¬ 
trast  measures  require  the  least  computation  since  a  series 
of  compare  operations  will  yield  the  necessary  maximum  and 
minimum  subimage  density  values.  The  Cramer-Rao  bound  re¬ 
quires  the  most  computation  since  we  must  compute  either  the 
pericdogram  or  the  autocorrelation  function. 

The  limiting  form  of  the  Cramer-Rao  measure  given  by 
(3-22),  however,  can  be  computed  optically  without  digitiz¬ 
ing  the  source  data.  This  is  accomplished  by  using  coherent 
illumination  and  appropriate  lenses  to  create  the  optical 
Fourier  transform.  A  filter  which  weights  each  frequency 
domain  component  by  the  radial  component  of  the  spatial 
frequency  is  placed  in  the  frequency  plane  and  the  resulting 
irradiance  distribution  is  sensed,  integrated,  and  recorded. 
This  energy  represents  tne  second  moment  of  the  subimage 
power  spectrum.  An  aperture  which,  provides  a  pre-transform 
windowing  function  can  also  be  provided.  A  quick  calcula¬ 
tion  involving  the  noise  power  estimate  then  yields  the 
Cramer-Rao  measure.  It  should  be  mentioned  that  a  limiting 
factor  in  any  conerent  optical  computation  is  the  so-called 
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"speckle  effect"  [Considine  1966]  which  is  a  consequence 
of  the  fact  that  the  ini  pulse  response  function  of  the  opti¬ 
cal  system  is  complex  which  allows  for  destructive  inter¬ 
ference  which  would  not  occur  in  an  incoherent  system.  It 
is  partly  because  of  this  effect  that  incoherent  optical 
spatial  frequency  analysis  techniques  are  being  developed 
[Cole  1930]  ,  [Rhodes  1980]  . 

The  variance  of  a  subsection  can  also  be  sense a 
optically  by  coherently  illuminating  the  desired  subsection, 
blocking  "dc"  in  the  frequency  plane  and  integrating  over 
the  Fourier  irradiance  distribution.  Given  the  variance  and 
the  estimated  noise  power,  the  quality  measure  Q  can  be 
easi ly  computed. 

The  optical  measurement  of  the  contrast  measures  is 
not  so  straightforward.  Recent  advances  in  non-linear  opti¬ 
cal  image  processing  have  resulted  in  feedback  systems  using 
Fabry-Perot  interferometers  containing  phase  recording  media 
[Ba rtheoi emew  and  Lee  1980].  At  the  time  of  this  writing, 
these  systems  are  capable  of  performing  optical  image  tnresn- 
olding  and  analog  to  digital  conversion  with  a  resolution  of 
about  3  gray-levels  [Atkins  et  al.  1980]  which  does  not 
appear  to  be  sufficient  for  the  application  at  hand. 
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Relief  Distortion 

In  Chapter  5,  a  model  was  developed  which  describes 
the  geometric  distortion  between  stereo-pair  images  as  a 
function  of  the  terrain  topography.  The  effect  of  this  dis¬ 
tortion  on  the  correlation  process  was  shown  to  be  a  reduc¬ 
tion  in  the  peak  height  of  the  correlation  function  with  re¬ 
sulting  increases  in  local  registration  and  false-acquisi- 
tion  errors.  This  behavior,  however,  was  shown  to  be  inde¬ 
pendent  of  the  statistics  of  the  local  image  subsection  in 
a  manner  described  by  the  distortion  measure  given  in  (5-27). 
Simulations  described  in  Chapter  5  verified  the  nature  of  the 
distortion  measure.  The  use  of  such  a  measure  as  an  error 
predictor  is  not  straightforward  however,  since  the  distri¬ 
bution  of  the  linear  distortion  parameter  a  is  difficult  to 
obtain  without  performing  the  s tereo- comp i 1  a t i on  process. 

An  attempt  at  estimating  the  statistics  of  a,  however,  led 
to  an  "image  overlay  quality"  measure  given  by  Equation 
(5-58).  The  utility  of  such  a  measure  in  the  prediction  of 
correlation  behavior  is,  at  this  point,  uncertain. 

The  most  promising  approach  for  dealing  with  relief 
distortion  in  a  pre - proces s i ng  mode  is  to  compute  a  map  of 
the  image  signal  bandwidth  to  be  combined  with  terrain  slope 
information  acquired  during  the  correlation  processing  to 
yield  an  estimate  of  the  distortion  measure  in  locations 
where  the  bandwidth  exceeds  the  constraint  specified  in 
Equation  (6-6).  Geometric  correction  could  be  applied  at 
these  points  with  well-known  procedures. 


Image  Enhancement  for  Correlation 
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In  this  section,  we  present  three  procedures  for 
suppressing  additive  noise  which  may  find  an  application  in 
improving  correlation  accuracy.  Since  the  statistics  of  an 
image  are  usually  non-s tationary ,  both  spatially  variant  and 
spatially  invariant  techniques  are  considered. 

The  first  approach  involves  simple  low-pass  filtering 
using  a  spatial  averaging  filter.  The  motivation  for  this 
procedure  derives  from  considering  the  effect  of  low-pass  fil 
tering  on  the  correlator  input  SNR.  Let  { x -j  ,  x x^  }  repre 
sent  a  sequence  of  samples  from  a  stochastic  process  and  let 

1  N 

W^xi  (7-2) 

represent  the  averaging  process.  If  we  assume  that  the  { x . } 

—  2 

each  have  mean  x  and  variance  jx  then  it  follows  that 


and 


y  =  Ely}  =  E { x }  =  x 
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where  C x ( i  ,  j  )  =  E 
covariance  matrix 
correlated  noise, 


(xi -x ) (Xj-x)  is  the  (i,j)th 
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If  we  now  let  £  represent  the  underlying  image  data, 
then  the  signal- to -noise  ratio  of  the  filtered  image, 

SNRf ,  is  given  by 


SNRf 


N  N 

L  L. 

i=l  j  =  l 


Cg(1’j) 


(7-6) 


,  N  N 

=  SNR  +  — -p  l  l  C  ( i  ,  j )  (7-7) 

Na  i  =  1  j  =  l  9 
n  i/j 

Thus  if  tne  image  data  is  correlated,  there  can  be  a  net 
improvement  in  SNR  (this  can  also  be  easily  seen  by  consid¬ 
ering  the  effect  of  low-pass  filtering  in  the  frequency 
domain).  The  fractional  improvement  in  SNR  is  given  by 


SNR  -SNR 
SNR 


1  N  N 

Nag  i=1  j*l 
9  1/j 


( i  ,  j  ) 


(7-8) 


Equation  (7-8)  implies  that  there  may  exist  an 
optimum  value  of  N  which  depends  only  on  the  form  of 
C Q .  Thus  if 
N 

T  [C(i,N  +  l)  +  C  ( N  + 1  ,  i  )  ]  >  a 
i  =  1  9  9  9 

we  can  obtain  further  improvement  in  SNR  by  increasing 
the  size  of  the  averaging  window.  It  is  also  possible 
that  (7-9)  can  be  continued  to  be  satisfied  for  all  N 


as  N  increases  without  bound.  Although  it  is  not  shown 
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explicity,  (7-9)  can  be  used  to  determine  the  optimum 
shape  and  orientation  (if  one  exists)  of  the  averaging 
filter  as  well  as  the  total  number  of  points  since  the 
spatial  proximity  of  points  in  the  averaging  window  will 
have  a  bearing  on  the  values  of  the  elements  in  C  .  The 
geometry  of  the  filter  which  maximizes  the  SNR,  however, 
does  not  necessarily  minimize  the  correlation  M  S  £ ,  par¬ 
ticularly  if  the  correlation  process  is  carried  out  along 
epipolar  lines.  For  example,  a  24-point  filter  can  have 
a  variety  of  shapes  and  orientations  (2x12,  12x2,  4x6, 
etc.).  Smoothing  in  the  direction  parallel  to  epipolar 
lines  will  reduce  the  false  acquisition  probability  (due  to 
improved  SNR)  at  the  expense  of  an  increase  in  local  regis¬ 
tration  errors  due  to  the  smoothing  of  the  resulting  cor¬ 
relation  functions.  Since  local  registration  errors  tend 
to  be  smaller  than  false  acquisition  errors,  this  effect 
may  be  desirable.  Furthermore  the  reduction  in  signal 
bandwidth  produced  by  such  filtering  results  in  reduced 
sensitivity  to  relief  distortion.  On  the  other  hand, 
smoothing  in  the  direction  perpendicular  to  the  epipolar 
lines  will  result  in  improved  SNR  without  drastically 
smoothing  the  correlation  functions. 

Because  of  the  spatial  warping  due  to  relief,  how¬ 
ever,  points  in  one  image  lying  along  a  line  per  pend  i  c u  1  a r 
to  an  epipolar  line  are  not  necessarily  colinear  in  the 
other  image.  Thus  averaging  data  in  the  cross-epi polar 
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direction  can  reduce  the  similarity  (correlation  coeffi¬ 
cient)  between  the  filtered  signals.  With  these  considera¬ 
tions  in  mind,  it  is  clearly  more  difficult  to  specify  the 
optimum  window  geometry. 

A  further  limitation  is  imposed  on  the  filtering 
process  by  the  quantization  required  for  subsequent  digital 
processing.  Low-pass  filtering  reduces  the  dynamic  range 
of  the  signal  which  decreases  the  correlator  output  signal- 
to -qu a n t i za t i c n  noise  ratio  as  well  as  the  signal-to¬ 
rn  a  c  h  i  n  e  noise  ratio  (see  Equations  (4-31)  and  (4-82).  Thus 
excessive  filtering  can  result  in  inferior  correlator  be¬ 
havior. 

An  experiment  was  performed  to  observe  the  effect 
of  low-pass  spatially  invariant  filtering  (using  averaging) 
on  the  accuracy  of  the  normalized  covariance  stereo- 
compilation.  Table  (7-1)  displays  the  MSE  resulting  from 
tne  correlation  process  as  a  function  of  the  geometry  of 
the  averaging  window.  From  these  results,  it  is  clear  that 
significant  improvement  in  MSE  (on  the  order  of  732)  can 
be  achieved  by  simple  low-pass  filtering.  These  results 
also  imply  tnat  orientation  is,  indeed,  a  significant 
factor.  Note  that  better  results  are  obtained  by  smoothing 
primarily  in  the  direction  perpendicular  to  the  epi polar 
search. 

Finally,  we  note  that  (7-2)  can  be  generalized 
to  provide  for  more  arbitrary  filtering  by  allowing  a 
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weighting  sequence  • w .  }  so  that 

1 

y  =  N  I  w i x i  •  (7-10) 


The  second  approach  to  noise  suppression  is  based 
on  the  common  enhancement  procedure  known  as  unsharp  mask¬ 
ing  [Pratt  1978].  This  procedure  consists  of  subtracting 
a  blurred  version  (the  "mask")  of  the  original  from  the 
original,  and  then  adding  the  weighted  difference  image 
back  to  the  original.  The  process  can  be  described  by  the 
relation  [Gray,  Hunt,  and  McCaughey  1979] 

Y  =  G  [  X -M  (  X  )  1  +  'x M (  X  )  (7-11) 


where  X  is  the  original  image,  M  ( X )  is  the  mask  of  the 
original,  G  is  an  arbitrary  gain  function  which  may  be 
spatially  variant,  and  x  is  a  fixed  parameter  which  weights 
the  mask  image  with  respect  to  the  difference  image.  For 
the  present,  we  let  x  =  1  and  define  5  by 


if  a2  >  k-2  (7-12) 

x  n 

otherwise 

where  ^  is  the  measured  local  variance  and  k  is  an  "adjust¬ 
ment  factor'  which  adds  some  flexibility  to  the  process.  In 
regions  dominated  by  noise,  the  output  image  approaches  the 
mask,  whereas  in  high  SNR  regions,  the  output  approacr,  tne 
original.  A  preliminary  experiment  was  performed  in  which 
the  maskirg  function  was  simply  an  averaging  filter  which 
replaces  the  center  pixel  of  an  MxM  window  with  the  mean 
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value  of  the  window.  Results  are  shown  in  Table  7-2  as  a 
function  of  the  parameter  k  for  M  =  3  and  M  =  5.  As  k  -*■  * , 
this  method  approaches  simple  averaging.  The  results  from 
the  invariant  averaging,  however  imply  that  improved  results 
may  be  achieved  by  a  judicious  choice  of  window  geometry. 
Comparison  of  identical  geometries  between  the  methods 
implies  t.nat  there  is  no  advantage  to  unsharp  masking. 

Finally  we  consider  a  procedure  known  as  short 
space  spectral  subtraction  which  has  been  recently  develop¬ 
ed  [ L i m  1  9 3 0 i  and  has  shown  considerable  promise  as  a  noise 
cleaning  scheme  which  is  performed  prior  to  the  application 
of  image  restoration  procedures  (e.g.,  deblurring).  This 
approacn  consists  of  sectioning  the  original  image  into 
overlapping  blocks  (16x16  pixels),  windowing  the  individ¬ 
ual  subsections  and  removing  the  estimated  noise  by  power 
spectral  subtraction,  a  procedure  wherein  only  the  ampli¬ 
tude  of  the  Fourier  components  are  altered  according  tc 
the  estimated  noise  spectrum  which  is  weighted  by  a  factor 
k  similar  to  Equation  (7-12).  The  filtered  blocks  are 
then  reassembled  to  form  the  cleaned  image.  Results  are 
tabulated  in  "able  7-2  for  several  values  of  k. 

Although  all  three  noise  suppression  procedures 
provide  reduction  in  correlation  MS E  ,  the  most  encouraging 
approach,  from  both  computational  complexity  and  resulting 
MSE  viewpoint,  is  the  sample  low-pass  filter.  Spectral  sub¬ 
traction  is  certainly  the  most  complex  in  terms  of  imple¬ 
mentation  and  does  not  provide  the  flexibility  available 
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with  the  other  methods.  As  stated  previously,  further  re¬ 
ductions  in  MSE  can  probably  be  obtained  by  optimizing  the 
window  geometry  and  filter  weights  for  the  low-pass  filter¬ 
ing  technique.  By  observing  the  histograms  of  the  correla¬ 
tion  errors  before  and  after  the  filtering,  it  can  be  seen 
that,  regardless  of  the  technique,  there  is  a  trade-off  be¬ 
tween  false  acquisition  and  local  registration  errors.  In 
other  words,  the  filtering  reduces  the  frequency  of  the 
relatively  large  errors  at  the  expense  of  a  small  (<  5  % ) 
reduction  in  correct  acquisitions  and  an  increase  in  small 
error  ( parti cul arly  1  pi xel -errors )  frequency. 
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Table  7-2.  Residual  MSE-Unsharp  Masking 
and  Short  Space  Spectral  Sub- 


traction  Noise 
Unfiltered  MSE 

Suppression, 

=  .8659 

Method 

k 

Residual  MSE 

Unsharp  Masking 

1 .0 

.8909 

(5x5  mask ) 

2.0 

.6421 

3.0 

.6133 

Unsharp  Masking 

3.0 

.5928 

(3x3  mask) 

4.0 

.5537 

6.0 

.  5261 

Short  Space 

1  .92 

.7410 

Spectial  Subtraction 

2.56 

.  71  23 

3.20 

.6997 
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Extension s 

The  experimental  results  presented  herein  are  based 
on  the  properties  of  the  synthetic  images  shown  in  Figure 
2-5.  While  there  is  no  reason  to  believe  that  the  results 
are  not  representative  of  the  nature  of  the  correlation  pro 
cess  and  its  pred i c ta b i 1 i ty  ,  there  are  obvious  shortcomings 
which  would  require  extensive  simulation  work  to  alleviate. 
To  begin,  the  accuracy  of  the  correlation  processors  is  de¬ 
pendent  on  the  data  acquisition  geometry  and  while  we  have 
included  this  factor  in  the  form  of  the  base-height  ratio 

g 

(rr-).  there  are  trade-offs  involved.  As  an  example,  we  not 
H0 

from  Equation  (1-3)  that  an  increase  in  base-height  ratio 
results  in  improved  elevation  resolution  but  from  Equation 
(5-12)  we  see  that  the  increase  in  elevation  resolution 
comes  at  the  expense  of  increased  distortion  which  degrades 
the  correlator  performance.  It  would  be  interesting  to 
consider  cnis  trade-off  in  more  detail  to  determine  if 
optimum  conditions  can  be  specified. 

The  correlator  models  developed  in  Chapter  4  pro¬ 
vide  information  regarding  the  necessary  degree  of  machine 
precision  in  order  to  relegate  machine  noise  as  a  second 
order  error  source.  Further  simulation  work  is  required  to 
verify  these  models  for  both  fixed  point  and  floating  point 


process i ng . 
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The  detection  of  sources  of  edge-related  errors  was 
based  on  the  gradient  of  the  variance  map.  It  is  clear 
from  the  results  that  improvement  in  the  detection  capabil¬ 
ity  will  require  a  look  at  the  form  of  the  individual  image 
data,  sequences  with  respect  to  the  form  of  the  edge  itself. 
However,  it  is  preferable  that  such  a  processor  not  be  re¬ 
quired  to  use  correlation  as  a  similarity  measure  (due  to  the 
computational  load).  In  Chapter  4,  we  presented  two  approach¬ 
es  to  the  detection  of  such  regions  (variance  gradient  and 
and  variance  outlyer)  and  it  is  conceivable  that  a  combina¬ 
tion  of  these  approaches  in  a  pattern  recognition  mode  would 
yield  an  improvement  in  detection  capability. 

The  use  of  classical  pattern  recognition  schemes 
(Fukunaga  1972],  (Kasdan  1971]  in  which  a  variety  of  fea¬ 
tures  are  combined  in  the  detection  processes  has  been  con¬ 
sidered  in  the  context  of  correlation  prediction  [Ryan,  Gray, 
and  Hunt  1930].  In  such  a  procedure,  feature  values  as¬ 
sociated  with  known  correlation  behavior  ( M  S  E )  are  input 
to  a  pattern  recognition  package  in  a  "learning"  mode. 

The  pattern  recognition  algorithms  rank  and  weight  each 
feature  to  maximize  the  detection  statistics  over  the 
known  behavior.  The  weightings  can  then  be  used  in  the 
detection  mode  on  data  for  which  the  correlation  behavior 
is  to  be  assessed.  Preliminary  investigations  in  this 
area  using  the  features  developed  herein,  have  indicated 
that  little,  if  any,  improvement  in  detection  capability 
is  gained  by  such  a  combination,  presumably  because 


all  the  f t-atures  are, 


qualities 
however,  i 
multiple  f 
for  such  a 


in  effect,  measuring  the  same  image 
(texture  or  contrast).  Further  work  in  this  area 
s  certainly  worthwhile  although  the  acquisition  o 
eatures  as  well  as  the  required  software  overhead 
recognition  capability  must  be  considered. 
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APPENDIX  B 


DERIVATION  OF  EQUATION  (5-17) 


We  begin  with  Equation  (5-16): 


D 

2 

E(rU)  )  =  |  Cf  (6  +  x  -  h(x)  )dx 
D 

'2 

If  u(x)  =  h ( x )  -  x  is  a  monotonic  function 

there  exists  a  function  z  with  z ( u )  =  x.  With  this 
of  variables,  (s-1)  becomes 


E(r(0) 


l  /  D  \  D 
h  (  2  | "  2 


z  (u)Cf ( •  -  u ) du 


Since  u(x)  is  monotonic  then  either  du/dx  > 
du/dx  <  0  for  all  xe[-j,  |] .  But 


Thus  if  du/dx  >  0,  then  z  (u)  >  0  and 
h  ( 2 )  "  2  >  2>  2 

so  that 

h(f)-f 

E ( r( c ) )  =  |  i z' (u ) j  Cf ( S  -  u  )du 

h(-f)+| 


(B-l  ) 

)f  x  ,  then 
change 

.  (B-2) 

0  or 

(  B  -  3 ) 

( B-4 ) 

( B-5 ) 
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